Skip to content

perf(update): apply same-size format-2 sparse patches in place - #273

Merged
peters merged 3 commits into
mainfrom
autoresearch/in-place-bspatch
Sep 1, 2026
Merged

perf(update): apply same-size format-2 sparse patches in place#273
peters merged 3 commits into
mainfrom
autoresearch/in-place-bspatch

Conversation

@peters

@peters peters commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Purpose

Autoresearch round 10 on the auto/update/ surface (series: #252#256#265#270). Phase measurement showed the per-step bspatch (~0.9–1.2 s at benchmark scale) still reads the whole 1.06 GB source and writes a full temp copy even when one 64 MiB chunk (one 4 KiB page) changed — the format-2 identity bitset already tells the apply side which chunks are unchanged, but it only used them to skip the diff, not the write.

Behavior impact

  • None observable on success or failure paths that matter. For same-size format v2 patches the file is patched in place: unchanged chunks are untouched, changed chunks are rewritten at their existing offsets (chunk boundaries align 1:1 at equal sizes → reconstructed file byte-identical to the temp-file flow). The target SHA-256 is one full read of the patched file afterwards — the step stays fail-closed (mismatch → same error message; tamper regression test unchanged).
  • Size-changing patches and format v1 patches take the temp-file flow, exactly as before.
  • The chain-walk tree is scratch space: the installed app is only touched by the separate atomic swap, so in-place patching on a failed step leaves no observable state (the tree is discarded either way).
  • chunked_bspatch_file* public API is unchanged; new additive chunked_bspatch_file_with_progress_and_sha256_in_place + ChunkedBspatchResult. Apply side split into chunked/bspatch.rs (600-line guardrail).
  • Harness fixes (first commit, needed for a valid baseline after Quiesce installed app before update swap #255/Write chunked patches in format 1 by default; format 2 is opt-in #267 landed):

Test evidence

  • RUSTFLAGS="-D warnings" cargo test --workspace — all green; new unit tests: in-place round-trip (hash + bytes + output-path untouched), all-identity no-op, size-change fallback, legacy-format fallback.
  • clippy (all-targets/all-features + strict panic paths), rustfmt, maintainability — all clean.
  • Same-session A/B (scale 1.0, seed 42, 48-core shared host), base = this branch minus the code commit:
    • Phase-instrumented 10-step pairs (×3): bspatch phase ~870/1,177/964 ms → ~648/648/587 ms per step; in-place path fired 30/30.
    • 100-delta wall clock: 147,030 / 161,507 ms vs 163,096 / 170,127 ms (−7.4% median; new < base in all 4 pairings).
    • Install tree byte-identical (bench asserts; output_size 1,214,024,073 on both sides); download bytes unchanged (503,524 B).
  • Bonus measurement (recorded, no change): the per-step repack is at the zstd-3 MT-48 floor — publisher-side pack and client-side repack cost the same (~500–550 ms stable); the 0.6–3.0 s spread is host load, not algorithmic waste. Frame reuse would need a per-entry archive format change.

Two harness corrections after #255/#267 landed:
- #255 makes the update manager refuse to swap an active application
  without its locally persisted process identity; the bench installed
  the baseline by raw archive extraction, which never writes the
  runtime manifest. Write it the way a real install would.
- #267 made format 1 the default chunked patch format (safe for
  pre-v2 readers); the canonical update baseline must pin
  chunked_patch_format: 2 to stay on the identity-chunk apply path
  established by the round 7-9 measurements.
Same-size format version 2 patches (identity-chunk bitset) rewrite only
the changed chunks at their existing offsets instead of reading the
whole source and writing a full temp copy; the target SHA-256 is one
full read of the patched file afterwards. Chunk boundaries align 1:1 at
equal sizes, so the reconstructed file is byte-identical to the
write-to-temp flow; size-changing patches and format 1 patches fall
back to the temp-file flow unchanged. The working tree is scratch
space for the chain walk (the install swap is a separate atomic step),
and the full-read target hash keeps the step fail-closed, so
verification semantics are unchanged.

Same-session A/B (scale 1.0, seed 42, 48-core shared host):
- phase-instrumented 10-step pairs (x3): bspatch phase
  ~870-1,180 ms -> ~590-650 ms/step, in-place path fired 30/30
- 100-delta wall clock: 147,030/161,507 ms vs 163,096/170,127 ms
  (-7.4%; new < base in all 4 pairings); install tree byte-identical
- unit coverage: in-place round-trip, all-identity no-op,
  size-change fallback, legacy-format fallback
@peters
peters merged commit 6c8a834 into main Sep 1, 2026
15 checks passed
@peters
peters deleted the autoresearch/in-place-bspatch branch September 1, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant