perf(gc): predict the futile full mark-sweep instead of pricing it afterwards (retain 3.99x -> 2.54x node, retain_wide 7.01x -> 2.89x) - #7799
Draft
proggeramlug wants to merge 8 commits into
Conversation
…les futile full mark-sweeps retain.ts 0.542 -> 0.345 s, retain_wide.ts 1.099 -> 0.454 s, deeplist.ts 0.245 -> 0.123 s, with peak RSS DOWN on all three. Quiet M1 mini, best-of-5. These programs retain every record they allocate, so nothing is ever garbage, yet 79% of retain and 88% of retain_wide was GC pause -- dominated by full mark-sweeps that found the heap fully live (retain 161 ms for 11.9%, retain_wide 98 + 512 ms for 6.8% and 9.6%, deeplist 127 ms for 0.0%; against tree/tree_wide's 40 fulls each at 87.8%/92.3%, which this leaves untouched). The escalation rule was "run a full once the arena grows past 2x the last full's live set". That is right for a heap accumulating garbage and wrong for one that is not: when everything allocated stays alive, doubling is the program working. #7726/#7733's retrospective backoff cannot repair it -- it prices a full after paying for it, and on a monotonically growing live heap deferring a full only makes the next one bigger. The futile full has to be predicted. The prediction is a measurement the collector already takes. young_survival_permille separates the populations by two orders of magnitude with nothing between: churn/churn_alloc/push_cls 0-4, cycles 0, shapes 713-920, retain/retain_wide/deeplist 999-1000. A copying minor at or above 900 permille marks the heap RETAINING, which widens the escalation growth band 4x and re-baselines arena-growth pacing on the occupancy that survived -- the latter is what makes the former reachable, since before the first full the baseline is 0 and the boundary degenerates to the absolute floor, so ANY program retaining more than 32 MB paid a whole-heap mark-sweep for doing so. The same signal and multiplier apply to old_reclaim_pressure_due's growth band. credit_promoted_bytes_to_old_baseline (#7592) already exempts old-gen growth a minor proved live, but a large object is allocated straight into old-gen and never passes through promotion, so its bytes are uncredited growth even when they are the program's live data -- on retain.ts, the element array itself. With the arena-growth escalation correctly declining, that band became the binding constraint and fired a 452 ms full reclaiming 7.6%. Two bounding properties, both asserted by tests: the baseline only ratchets up and the multiplier is >= 1, so the boundary is never lower than before and this can only make fulls rarer, never more frequent; and one non-retaining minor disarms the band with no decay window. `retaining` is emitted in the major_pacing GC trace so a run that never armed it is distinguishable from one that did and had nothing to skip. Two independent wins found while profiling: * An all-pointer array's dirty-card scan was O(live array), not O(dirty pages). scan_dirty_object_slots's Slot arm answers "is this slot dirty?" with a hash-set probe per slot; its Range arm intersects with the dirty-page set directly. LayoutSlotMask::AllPointers reported itself as Masked and so emitted one Slot per element -- 3M probes per minor to find a few hundred known-dirty pages. dirty_slot_ranges_scanned == 0 in every retain.ts trace was recording exactly that. Worth 9% on retain before any pacing change. dirty_slot_ranges_for now also walks whichever of the two sets is smaller. * classify_heap_space_in_range is split into an inline(always) cache-hit arm and an inline(never) miss arm, as #7469 did for classify_heap_generation and for the same reason; and classify_arena no longer reads both survivor-space thread-locals (two _tlv_get_addr calls on Darwin) before a match whose common arms cannot use them. Refuted and not shipped: batching the per-slot old_page_account_dirty_slot map probe into one update per 4 KB page measured as exactly zero (0.344 vs 0.345 s).
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
added 7 commits
August 10, 2026 23:11
…t survivor placement gc-ratchet 11_collect_at_depth turned 6,150 promoted objects into 6,139 copied ones: copied_minor_promotion_handoff_pressure_due shares old_reclaim_pressure_due with the OldReclaim escalation, so widening the shared band also stopped the survivor-promotion handoff from firing on a retaining heap. Placement and collection are different questions and only the second one was paying for a futile full, so the multiplier moves to old_reclaim_full_due and the shared band goes back to what it was. Pinned by a test that asserts both directions from one reading.
…sion, not survivor placement" This reverts 882be57. Both of its justifications were refuted by measurement. The premise was that widening the shared band flipped gc-ratchet's `11_collect_at_depth` from 6,150 promoted objects to 6,139 copied ones. It did not: a gc_ratchet run of the `origin/main` @ 0a2bf15 reference build on the same host produces that flip too, along with `04_dead_after_deep_stack`'s copied_objects row -- six identical gating rows, byte for byte. Comparing the two artifacts cell by cell, EVERY gating metric across all 13 probes is identical between main and this branch; only wall_ms/rss_bytes/peak_rss_bytes differ, and those are the three the shared_ci profile deliberately does not gate. Those rows are red on main on this host, not something this PR did. The principle behind it was wrong too. It carved `copied_minor_promotion_handoff_pressure_due` out as a survivor-PLACEMENT decision that should not read a band derived from full-GC-yield evidence. Its own doc says otherwise -- "whether an imminent promotion justifies a full old reclaim FIRST" -- and `gc::mod.rs` responds to it with `note_survivor_promotion_handoff_full`. It is a full-collection decision like the other two, so one signal and one multiplier across all three callers of `old_reclaim_pressure_due` is the coherent shape, not a carve-out.
…ng gc-ratchet rows
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the largest remaining deficit on the
retainfamily. On the quiet M1mini, best-of-5, against
origin/main@0a2bf15bd:retainretain1retain_wideretain_wide1deeplist(not targeted)Peak RSS fell on all of them (
retain332.4 → 331.3 MB,retain_wide466.4 → 453.9 MB,
deeplist103.4 → 99.2 MB), because the full collections thisremoves were reclaiming 0–12%.
The finding
retain.tsbuilds a 3M-element array of records and keeps every one alive.Nothing is ever garbage. 79% of its runtime was GC pause (88% for
retain_wide), and the pause was dominated by full mark-sweeps that found theheap fully live:
retainone full at 161 ms reclaiming 11.9%,retain_widetwoat 98 ms and 512 ms reclaiming 6.8% and 9.6%,
deeplistone at 127 msreclaiming 0.0%. Against
tree/tree_wide, whose 40 fulls each reclaim87.8% / 92.3% and which this PR leaves untouched.
The escalation rule was "run a full once the arena grows past 2× the last full's
live set". That is right for a heap accumulating garbage and wrong for one that
is not — when everything allocated stays alive, doubling is the program working.
#7726/#7733's retrospective yield backoff cannot repair it: it prices a full
after paying for it, and on a monotonically growing live heap deferring a full
only makes the next one bigger. The futile full has to be predicted.
The change
The prediction is a measurement the collector already takes.
young_survival_permilleseparates the two populations by two orders ofmagnitude with nothing in between —
churn/churn_alloc/push_cls0–4,cycles0,shapes713–920,retain/retain_wide/deeplist999–1000. Acopying minor measuring ≥ 900 permille marks the heap RETAINING, which widens the
escalation growth band 4× and re-baselines arena-growth pacing on the occupancy
that survived. The same signal and multiplier are applied to
old_reclaim_pressure_due's band, which became the binding constraint once thearena-growth escalation correctly declined (a 452 ms full reclaiming 7.6% — the
identical shape, one trigger over).
Two properties bound the blast radius, both asserted by tests:
ratchets up and the multiplier is ≥ 1, so the boundary is never lower than
before. The exposure is deferred reclamation, and measured RSS went down.
Plus two independent wins found while profiling: an all-pointer array's dirty-card
scan was O(live array) rather than O(dirty pages) (worth 9% on
retainalone,and
dirty_slot_ranges_scanned == 0in every trace was recording it), and twohot-path readings in the copying minor that were paid for and not used.
Full writeup, including the hypothesis that measured as exactly zero and was
therefore not shipped, in the changelog fragment.
Validation
node --experimental-strip-types, exit 0.gc-handoff/apps/iso_miss.tsprintschecksum 437840 misses 0, including underPERRY_GC_PROTECT_FROMSPACE=1 PERRY_GC_PROTECT_FROMSPACE_DEPTH=800andPERRY_GC_VERIFY_EVACUATION=1.tree/tree_widestill run their 40fulls each at the same wall time.
perry-runtimelib suite green (2,054 tests,RUST_TEST_THREADS=1).main.11_collect_at_depth(six gating rows)and
04_dead_after_deep_stack'scopied_objectsfail identically for theorigin/main@0a2bf15bdreference build measured on the same host. Comparingthe two artifacts cell by cell, every gating metric across all 13 probes is
identical between main and this branch; only
wall_ms/rss_bytes/peak_rss_bytesdiffer, which is exactly what theshared_ciprofile does notgate.
12_large_live_setholds and improves (heap_used_bytes−2.39%) withcopied_objects61,851, so its subject was live. Those rows want their owninvestigation and are not from this PR.