perf(update): skip redundant basis re-hash across sparse chain steps - #265
Merged
Conversation
The chain walker verifies every patched file's target SHA-256 at the end of each step; the next step's basis check then re-reads and re-hashes the very same content. Carry a VerifiedFileHashes map (path -> verified sha256) through apply_target_deltas: - PatchFile: basis verification is skipped when the cache records the expected hash (set by this walk's own target verification); otherwise the full basis check runs as before - WriteFile: target hash recorded after verification - Delete: entry dropped Safety: entries only ever come from verifications performed in the same walk, a missing/mismatching entry falls back to the full basis verification, and the post-patch target hash + per-step full-archive SHA-256 are unchanged, so external modification between steps is still caught (regression test corrupts the file between steps and expects the mismatch). Single-shot apply passes a fresh empty cache (no cross-step state).
…doc block The identity-chunk doc block from the #256 change was dropped when a multi-block edit was rejected wholesale; restore it and record the verified-hash carry numbers (100-delta apply 216.8 s -> 157 s, cumulative 657 s -> 157 s since the original baseline), the corrected per-step breakdown, and the updated open items.
This was referenced Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Autoresearch round 8 (update surface). Each chain step verifies the
patched file's target SHA-256; the next step's basis check then
re-reads and re-hashes the very same content. For the canonical sdk
step (one 4 KiB page in a 1.06 GB file) that is a ~0.55 s full-file
read + SHA-256 per step.
The chain walker now carries a
VerifiedFileHashesmap (path →verified sha256) through
apply_target_deltas:PatchFile: basis verification skipped when the cache records theexpected hash; first step and any mismatch still verify fully
WriteFile: target hash recorded after verificationDelete: entry droppedBehavior impact
hash-verified before it is used as a basis (just not twice); the
post-patch target hash and the per-step full-archive SHA-256 are
unchanged. A tamper-between-steps regression test corrupts the file
after step 1 (bypassing the cache) and asserts step 2 still fails
with a hash mismatch — verified the skip path is actually taken
(marker in the skip branch fired in the test).
docs/performance/update-chains.md, which was lost when amulti-block edit was rejected wholesale in the perf(diff): identity-chunk markers in chunked patches (format v2) #256 round, and
records the corrected per-step breakdown.
Test evidence
New regression test
test_in_place_chain_with_verified_cache_detects_external_modification;existing carried-chain equivalence test updated to use a shared
cache (asserts byte-identical output with caching on).
cargo test --workspacegreen; clippy all-targets + strict(
-D clippy::unwrap_used -D clippy::expect_used) clean; fmt clean;maintainability guardrail clean.
Same-session A/B, 100-delta chain, scale 1.0, sdk-only, seed 42
(baseline = main
bba9f4fin a separate worktree, interleaved run):−27.4% (0.9% spread); per step ~2.2 s → ~1.6 s; download bytes
(503,524 B) and install tree (1,214,024,029 B) identical across
all runs.
Full-chain history on this benchmark: 657 s → 489 s (perf(update): carry the extracted tree across sparse chain steps #252) →
221 s (perf(diff): identity-chunk markers in chunked patches (format v2) #256) → ~157 s (this change) — cumulative −76%, payload
byte-identical at every stage.