Skip to content

perf(update): stream the target SHA-256 into the bspatch write - #270

Merged
peters merged 3 commits into
mainfrom
autoresearch/streamed-target-hash
Sep 1, 2026
Merged

perf(update): stream the target SHA-256 into the bspatch write#270
peters merged 3 commits into
mainfrom
autoresearch/streamed-target-hash

Conversation

@peters

@peters peters commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Purpose

Autoresearch round 9 on the auto/update/ surface (continuing the
chain-apply optimization series #252#256#265). Phase
instrumentation in #256/#265 showed the per-step target SHA-256 is a
separate full read + hash pass after the bspatch write (~0.55 s
per step at the 1 GB benchmark scale). This change removes that
second read: the chunked bspatch write path now computes the
reconstructed file's hash while writing and returns it.

Behavior impact

  • None observable. Same payload, same verification semantics:
    the streamed hash covers exactly the bytes written; mismatch errors
    keep the same message; the tamper-between-steps regression test
    still fails closed; per-step repack + full-archive SHA-256
    unchanged.
  • API is additive: new chunked_bspatch_file_with_progress_and_sha256
    (+ ChunkedBspatchResult); the existing
    chunked_bspatch_file/chunked_bspatch_file_with_progress are
    untouched, so the archive strategy and all other callers are
    unaffected.
  • Cost model: the eliminated phase was read (I/O) + hash (CPU). The
    hash CPU is now paid inside the write loop, so the net win is the
    saved read (~0.2-0.3 s/step at benchmark scale — larger on
    slow/flash storage where the re-read dominates).

Test evidence

  • RUSTFLAGS="-D warnings" cargo test --workspace — all green
    (includes the new round-trip test asserting the returned hash
    equals the rebuilt file's SHA-256).
  • cargo clippy --workspace --all-targets --all-features clean;
    strict panic-path clippy clean; cargo fmt --all -- --check clean;
    scripts/check-maintainability.sh pass.
  • Same-session A/B (scale 1.0, 100 deltas, seed 42, 48-core shared
    host), interleaved base/new runs:
    • Phase-instrumented 10-step pairs (×4): the 540-563 ms/step
      target-hash phase is eliminated; per step ~2,270 →
      ~1,980-2,060 ms in 3 of 4 pairs (the 4th pair was a load-spike
      window that hit both sides' bspatch and repack).
    • 100-delta wall clock: new 186,122 / 167,182 / 157,514 ms vs
      base 178,113 / 166,808 ms. The ~0.2-0.3 s/step gain sits near
      this shared host's ~10% wall noise floor, so the claim rests on
      the phase-level evidence; documented in
      docs/performance/update-chains.md with the raw numbers.
  • Download bytes unchanged (503,524 B); install tree identical
    (bench asserts).

The per-step sparse-delta target hash was a separate full read + hash
pass after the patched file was written. chunked_bspatch_file now
accepts an optional Sha256 and chunked_bspatch_file_with_progress_and_
sha256 returns the reconstructed file's hash computed while writing,
removing that second full read per step (additive API; existing
functions unchanged).

Same-session evidence (scale 1.0, seed 42, 48-core shared host):
- phase-instrumented 10-step A/B, 4 interleaved pairs: the 540-563
  ms/step target-hash phase is eliminated; net per step
  ~2,270 -> ~1,980-2,060 ms in 3 of 4 pairs (4th pair was a
  load-spike window hitting both sides)
- 100-delta wall clock: new 186,122/167,182/157,514 ms vs base
  178,113/166,808 ms — the ~0.2-0.3 s/step gain is near the shared
  host's ~10% wall noise floor, documented as such
Verification semantics unchanged: the hash covers exactly the bytes
written, mismatch errors are identical, and the tamper-between-steps
regression test still fails closed.
Merge of #267 (format-1-by-default, format field on ChunkedDiffOptions)
left the round-trip test added by this branch with the old two-field
initializer.
@peters
peters merged commit 6b7e213 into main Sep 1, 2026
15 checks passed
@peters
peters deleted the autoresearch/streamed-target-hash branch September 1, 2026 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant