pgw#973 §4.24 wave 2: six safetensors header caps become one, plus three duplicate bounds and two false justifications

[PaulFidika]

Executes the tabled §4.24 verdicts from the pgw#973 census against current master. Every verdict was re-verified before acting — two were wrong in a way that would have removed a real bound, and both corrections are recorded rather than quietly worked around.

The headline: one threat, one number

A safetensors file opens with an 8-byte declared header length read straight from the file, and every reader turns it into json.loads(f.read(n)) before anything has validated it. That threat was stated six times: models/w4a4.py, models/w8a8.py, models/svdq.py, models/loading.py, convert/ingest.py at 100 MiB — and convert/writer.py at 512 MiB.

The outlier was a live defect, not just noise: the writer accepted headers the loader refuses, so the re-shard path could emit a shard the serving path could not open. Same bytes, two verdicts.

All six now read gen_worker.models.safetensors_header, where the threat, why nothing else prevents it (the length is read before any other structure exists), and why 100 MiB is a plausibility floor rather than a measurement — with what would change it — are stated once.

Surviving-bound proof (tests/test_one_safetensors_header_bound_pgw973.py): real files on disk through every real entry point, no mocks and no monkeypatching of the bound. The runaway (2**63-1), the over-cap, and the absent (0) length are refused at all six; a legitimate file still parses at all six; writer and loader now agree on a 200 MiB-declared header. A structural pin fails the suite if any module re-declares a header cap.

RED-verified: restoring 512 MiB in convert/writer.py fails test_writer_and_loader_agree_on_the_same_file — the writer accepts the length and only trips later on the short body, i.e. it would have attempted the 200 MiB read.

Also deleted / de-duplicated

• Dead: models/memory.py:_DEFAULT_VAE_SLICE_THRESHOLD_GB = 10.0 — zero references in src, tests, docs or scripts.
• Single owner: _READ_CHUNK_BYTES (4 MiB), defined identically in chunk_cas.py and chunk_upload.py — the latter already imports the rest of the chunk vocabulary from the former.

Two census corrections

1. input_assets vs url_fetch is NOT two caps on one fetch path. open_guarded_stream deliberately caps nothing ("the caller owns the read and its byte cap"), and url_fetch.fetch_bytes / input_assets._download_one are separate entry points that each enforce their own. Acting on the filed DELETE-REDUNDANT verdict literally would have left one path with no cap at all. The value is aliased to one owner instead.

2. _PRESIGNED_PUT_BUDGET is neither authoritative nor deletable. Its justification claimed "file-level fan-out is fixed at 4" and cited _concurrent_upload.py — a module that no longer exists. The only in-repo caller is sequential, so the file axis is 1 and optimal_part_concurrency's min(total_parts, 4) is what binds. The semaphore is KEPT because it covers the one axis optimal_part_concurrency cannot see: an endpoint author calling ctx.save() from their own threads. Both docstrings now say that instead of citing each other.

Not done here

_ARM_TIMEOUT_S's leader-progress-marker protocol change is already owned by pgw#892, which carries the exact task and a RED criterion — no new issue needed, cross-referenced instead. The remaining DELETE-NO-THREAT tranche is tabled in the tracker with per-bound verdicts, because several bounds the census swept in (_MAX_RESOLVE_BODY, _MAX_WALK_DEPTH, RENEW_FRACTION, _FINALIZE_TIMEOUT_S as a derivation basis) have real threats and are KEEP-BUT-DOCUMENT, not deletes.

Gates

Python 3.12. tests/ — 3297 passed, 37 skipped, 1 xfailed. The single -n 4 failure (test_procsplit_pgw763) passes in isolation and is untouched by this change (load-induced; box load was 17). tests/test_import_cycles_pgw981.py green — the new leaf module adds no cycle. Changelog fragment changelog.d/pgw973-wave2.md. Commit signature-verified.