issues: an image's GUIDs are drawn from the host's random source

[Japabu]

Files issues/an-images-guids-are-drawn-from-the-hosts-random-source.md. Issues only: no source, test or manifest changes.

Why

PR #873's metal images were staged from committed trees. When the AMD bench's session was cut off, the judges that ran afterwards found boots missing and rebuilt those images in place. Every rebuilt image came out with a different sha256 from the one the request named, including the ones rebuilt from an unchanged tree. The orchestrator asked why. ToyOS's north star is reproducing the host's bytes.

What two stagings of one tree differ in

Method. Six pairs, each the same selection staged twice by cargo test --test toyos-build -- --metal --metal-readback <dir> from one clean committed tree in one checkout, in two processes. Each pair was diffed byte by byte with a read-only script. The script maps every differing byte to a GPT region or partition, and inside each FAT32 volume to a file's contents or a directory-entry field. The scripts are diffs/imgdiff.py, diffs/pehdr.py and diffs/keyfind.py; diffs/commands.txt says which images each output compares, and diffs/exits.txt holds their exit codes, 0 each.

pair	tree	selection	bytes differing	kernel, loader, ROOT, LOG
p0	d4c2f8ebe	scout-tlb-base	396	identical
p1	d4c2f8ebe	latencycase	393	identical
p2	9b30f5e28	latencycase	411	identical
p3	9b30f5e28	scout-tlb-base	396	identical
p4	363e0e841	shared-debug	413	identical
p5	363e0e841	testcases	391	identical

Every differing byte has one of two sources, in all six pairs.

• GUIDs drawn from the host's random source. build/image.rs takes every GUID it writes from gptwrite::random_guid (getrandom), drawn again on each create_boot_image and lay call. Those bytes are the disk GUID in both GPT headers, five unique partition GUIDs in both entry arrays, the CRCs over them, /toyos/log.guid on the ESP, and slot A's two GUIDs in the slot table.
• The clock. These are version_now's signed version in /toyos/image.sig and the slot table, the signature over that version, and the FAT creation and write times (ctime, ctime_tenth, wtime) of every entry on the ESP and slot A's volume.

p1, counted from its per-region totals in diffs/p1-base-latencycase.txt:

• GUIDs: GPT headers 24 + 24, entry arrays 79 + 79, log.guid 16, slot table 32. That is 254.
• Clock: ESP entry times 44, slot A entry times 24, image.sig 65, slot table version 2. That is 135.
• The slot table's CRC over both: 4.
• 254 + 135 + 4 = 393, the pair's total.

Two checkouts of one commit (diffs/p6-head-two-checkouts.txt, diffs/p6-loader-pe.txt, diffs/keyfind.txt) differ further:

• 470 KB of the kernel and 18.7 MB of ROOT, which issues/two-checkouts-of-one-tree-build-different-guest-bytes.md records.
• 72 bytes of the loader. Two 32-byte runs are exactly the checkout's throwaway Ed25519 public key. That key is found twice in each image's own ESP and nowhere else in either image. The other 8 bytes are the PE header's and debug directory's TimeDateStamp, which are not clock readings (2073 and 2066 as Unix times).

A key per checkout is by design (issues/the-machine-updates-itself-without-ubuntu.md, "Keys"), an input like the config, and is not filed.

Decision: a new file, and which existing one holds the rest

• The clock half is issues/an-images-signed-version-and-its-fat-dates-are-the-hosts-clock.md, which names version_now and build_time. That file is not edited.
• Nothing filed held the GUIDs. I checked with git grep -n -i guid origin/main -- issues/. issues/the-loader-does-only-what-must-precede-the-handover.md states that one image's sticks share their GUIDs. issues/the-anti-rollback-floor-is-a-firmware-variable.md keys a throwaway floor on the log GUID. Neither is about reproducing the bytes.
• The new file therefore holds an image's GUIDs alone. It names the clock file for the other half, names the cross-checkout causes as someone else's, and records the constraint a fix inherits.
• Scope: every GUID an image carries, which create_boot_image writes. designate_data_disk and install draw a DATA GUID and, through lay, a disk GUID too, but for disks that are no image: the scratch disk create_sparse makes for a guest's /apps and /home (cargo run's target/nvme.img, the harness's NVMe file), and the disk a Storage::Disk test boots an installed image from (tests/common/qemu.rs). The scratch disk's only input beside its path is a length, so "a function of the image's own inputs" cannot be read for it. The file names both and says it does not hold them.
• The log GUID's readers, which the file names because a replacement must still give two images that differ two log GUIDs:
  • the floor: its variable's name hashes the log GUID (lib/update/src/floor.rs), so that no image the key signs holds another to a floor (build/signing.rs). This ends at stage 3 of issues/the-loader-does-only-what-must-precede-the-handover.md, which deletes floor::Scope and names every floor for its key alone.
  • the log-volume lookup (loader/src/loaderlog.rs).
  • the black box: a record's owner is the log GUID, and a record another image left is cleared rather than reported (loader/src/main.rs:685-688, loader/src/blackbox.rs harvest). This does not end: stage 9 moves the check into the kernel, still keyed on the log GUID.
  • the attempts record: it names its partition by the log GUID, and Record::decode refuses one naming another (loader/src/attempt.rs, lib/update/src/record.rs). This ends at stage 5, which replaces that record with the slot table's tries.
• Owner: issues/toyos-builds-itself.md. Its M4 exit is a guest build of the image whose hashes match the host build of the same commit, and no image whose GUIDs are drawn meets it. The file stays status: open, because that track is open and nobody holds it.
• Exit: every GUID create_boot_image writes, the disk GUID lay writes for it among them, is a function of the image's own inputs (its sections and its key), and a host test builds one image twice and finds its partition table, slot table and /toyos/log.guid identical. Stating the derivation keeps a GUID taken from the process id, the checkout path or the host's name from closing the file. The exit does not depend on the clock file, so either can close first.

Gates

gate	head	exit	log
cargo run -- --ci host	311c89a8c	0 (Host: 76 step(s), all green)	gates/311c89a8c/host.log, gates/311c89a8c/host.exit

No guest test is reached: the diff touches only issues/.

What I am unsure of

• The DATA and disk GUIDs designate_data_disk and install draw are read off the code, not measured: no metal image is either disk. The file leaves them out of its scope.
• Whether anything else relies on two builds of one plan carrying different GUIDs is not settled. git grep -n -i -E 'log_guid|log\.guid' -- tests/ build/ finds only build/image.rs. git grep -n -E 'floor::|floor_scope|ToyOSImageFloor|Scope::Image' -- tests/ build/ finds only the key's scope handed to the loader's build (build/build.rs, build/clippy.rs, build/signing.rs). The loader reads the log GUID as the floor's name, to find the log volume, as the owner of a black-box record and as the owner of the attempts record, and the file names all four. The loader also hands it to the kernel. The supervisor claims the log partition by that GUID, and the kernel refuses a claim when two partitions carry it (system/supervisor/src/main.rs, kernel/src/gpt.rs seek). The file does not name that reader. A metal bench keeps its UEFI variables across boots, so it holds a floor under every log GUID it has booted. Whether that matters once GUIDs stop being fresh is the fix's question, and I have not answered it.

🤖 Generated with Claude Code

https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy

[Japabu]

Review, round 1, head b2747ed30e6dfc07811df044357f86f90ba78f90, against .claude/agents/reviewer.md at origin/main cc52b456c.

Net (git diff --shortstat origin/main...b2747ed30): 1 file, +44 −0. No production code, no tests; all of it is in issues/.

Checked, no finding:

• Measurement (ToyOSOrg/evidence pr-881 at 26b7dd4ad, signed, commit message names this head):
  • In p0 to p5, the per-region totals add up to each pair's total (391 to 413), with 0 bytes outside every region. No pair has a kernel.elf or BOOTx64.EFI contents line, and ROOT and LOG are 0 in all six.
  • p1's 393 re-derives as 254 + 135 + 4.
  • exits.txt and keyfind.txt are 0 throughout.
  • The trees measured were d4c2f8ebe, 9b30f5e28 and 363e0e841. Each has build/image.rs, build/gptwrite.rs, build/signing.rs, lib/update/src/floor.rs and loader/src/loaderlog.rs identical to this head: git diff --quiet exits 0 for all three.
  • cargo run -- --ci host at this head: gates/host.exit is EXIT=0, and the log ends Host: 73 step(s), all green. Its 46 FAILED summaries are loom models that are meant to fail.
  • Every site the file names in build/image.rs (:239-241, :661, :710, :834, :602) is as the file states.
• Duplication: none.
  • The clock file holds only version_now and build_time.
  • The two-checkouts file holds checkout paths.
  • git grep -n -i guid origin/main -- issues/ finds no issue about reproducing an image's GUIDs.
• Identifying data: none, in the file, the body or the evidence.
  • No MAC, IPv4, IPv6, serial-number or personal-hostname shape.
  • The staging directory names give only a bench's kind.
  • keyfind.txt prints counts and offsets, never a key.

BLOCKER

• issues/an-images-guids-are-drawn-from-the-hosts-random-source.md:1-5,42-44: the file names no owner. The frontmatter says status: open, and neither the body nor the exit says who holds it.
  • Rules broken: reviewer.md, "Rank every finding" ("an issue … the branch writes without an owner that exists") and Growth ("recorded in issues/ with an owner, evidence and an exit condition"); root CLAUDE.md, "Zero silent debt".
  • Precedent: netring: a session region replaces the pipes between netstack and its clients (design, and the track that stages it) #863 was sent back on this same finding. The sibling filed the same day, issues/loom-arc-panics-in-its-drop-while-a-failed-model-unwinds.md, carries an owner line: "Owner: the orchestrator. Nobody holds it."
  • Fix: name an owner. One choice is issues/toyos-builds-itself.md, M4, since this defect blocks M4's exit (a guest build's image hashes match the host's). The other is the orchestrator.

NOTE

• issues/an-images-guids-are-drawn-from-the-hosts-random-source.md:42-44: the exit says what a GUID is not drawn from, but not what it is derived from.
  • A GUID taken from the clock, the process id, the checkout path or the host's name meets "no GUID build/image.rs writes is drawn from a random source".
  • All of those but the clock also pass a host test that builds the image twice in one process.
  • So the file can close while two builds of one plan still differ.
  • Fix: state the derivation, for example "every GUID build/image.rs writes is a function of the image's own inputs (its sections and its key)", so a close can be read against it.
• issues/an-images-guids-are-drawn-from-the-hosts-random-source.md:34-40: the file states the log-GUID constraint as permanent, but the record already removes the parts it rests on.
  • Stage 3 of issues/the-loader-does-only-what-must-precede-the-handover.md deletes floor::Scope and names every floor for the key alone.
  • Stage 2 of that file confines the loader's log-volume lookup to the boot disk.
  • The file names neither stage. A fix landed after them would keep two images' log GUIDs apart for a per-image floor that no longer exists.
  • Fix: name stage 3 as where the floor half of the constraint ends.
• issues/an-images-guids-are-drawn-from-the-hosts-random-source.md:19: the command as written, cargo test --test toyos-build -- --metal --metal-readback, is refused. --metal-readback takes a directory (tests/metal/src/testargs.rs:137), and :248 asserts that refusal. The command the six pairs were staged with named one. Fix: write --metal-readback <dir>. The PR body repeats the same command.

SEND BACK

[Japabu]

Review, round 2, head f72b74e13c58b7dc02545f433178bed546d67bca, against .claude/agents/reviewer.md at origin/main c0cdddf4e.

Net (git diff --shortstat origin/main...f72b74e13): 1 file, +50 −0. All of it is in issues/: no production code and no tests. Since round 1 the branch adds b0671d967 (+10 −4 in the same file) and a merge of c0cdddf4e. The merge adds nothing to the diff: git diff --stat origin/main f72b74e13 names only that one file.

Round 1's findings

• BLOCKER, the file names no owner: CLOSED.
  • :44-46 names issues/toyos-builds-itself.md.
  • That file exists at this head with status: open and kind: track, and its M4 exit (:25-26) reads as the file quotes it.
  • Nine other issues name the same track as their owner in the same form.
• NOTE, the exit gave no derivation: CLOSED. :48-50 now states one.
• NOTE, the log-GUID constraint was stated as permanent: CLOSED as asked.
  • :40-42 names stage 3.
  • At this head, issues/the-loader-does-only-what-must-precede-the-handover.md:124-126 says stage 3 deletes floor::Scope and names every floor for its key alone.
  • The new NOTE below covers what the constraint still rests on after that stage.
• NOTE, --metal-readback with no directory: CLOSED, in the file (:19) and in the PR body.

Checked, no finding

• Host gate. The evidence is ToyOSOrg/evidence at deed71480: its signature is verified and its commit message names f72b74e13.
  • gates/f72b74e13/host.exit is EXIT=0.
  • host.log was built in the branch's worktree and ends Host: 76 step(s), all green.
  • The log has no MAC, IP address, serial-number or personal-hostname shape.
• Guest tests. None is reached.
• Citations. Every path the file cites exists at this head. build/gptwrite.rs:111-113 is random_guid, which fills from getrandom.
• Issues added on main since round 1. None overlaps this file.
• The exit does not depend on the clock file, as the body says. create_boot_image takes the version in Signing (build/image.rs:171-174,231), so a host test can fix it.

BLOCKER

None.

NOTE

• issues/an-images-guids-are-drawn-from-the-hosts-random-source.md:48-49 — the exit's derivation ("a function of the image's own inputs (its sections and its key)") cannot be read for one GUID: designate_data_disk's DATA GUID, and the disk GUID lay writes for it (build/image.rs:648-661). :15-16 puts that site in scope, but it lays the harness's scratch disk (build/build.rs:1845), which carries no image. The PR body leaves "how the exit … reads for that GUID" to the fix, so whoever closes the file will read that GUID as unmeetable, or as met by any constant. Fix: state that GUID's derivation, or narrow :15-16 and the exit to the GUIDs of an image.
• issues/an-images-guids-are-drawn-from-the-hosts-random-source.md:34-42 — the constraint names only two things that need each image's log GUID to differ: the floor, whose end at stage 3 the file now states, and the volume lookup, which stage 2 confines to the boot disk. It leaves out the loader's black box, which takes the log GUID as the image's identity. loader/src/main.rs:685-688 says so ("build/image.rs mints one per image, so a record another image left in this memory is one this pass clears"), loader/src/blackbox.rs:140-151 acts on it, and stage 9 (issues/the-loader-does-only-what-must-precede-the-handover.md:413-414) moves that check into the kernel, still keyed on the log GUID. Once stages 2 and 3 land, the file gives no remaining reason for distinct log GUIDs while one remains, so a fix taken then could let one image report another's crash record as its own. Fix: name the black box beside the floor and the lookup.
• PR body, "What I am unsure of" — "The loader reads the GUID for the floor's name and the volume lookup" is false of the tree: the loader also reads it as the black box's identity (loader/src/main.rs:688,889) and as the attempts record's owner (loader/src/attempt.rs:26-79, lib/update/src/record.rs:123-125).

LAND AFTER NAMED CHANGES