Repository navigation
x86-64 kernel leaves are global under a declared CR4.PGE, and only a kernel-half change drops them - #873
x86-64 kernel leaves are global under a declared CR4.PGE, and only a kernel-half change drops them#873Japabu wants to merge 5 commits into
Conversation
|
Design acceptance, recorded by the orchestrator for the owner. He answered "Accept, build" to this design. On the KPTI question in the roast: no KPTI issue exists on |
…s already are The design accepted on #873, built. A leaf is global if and only if it is the same in every root that will ever be loaded, which on x86-64 is the kernel half: every 2 MiB leaf `paging::init` and `map_mmio` write, and the 4 KiB PTEs `guard_4k` splits out of one, now carry PAGE_GLOBAL, and only `kernel_leaf` writes it. No user leaf and no upper-level entry carries it: `Prot::leaf_bits` never names it and `ensure_table` masks to TABLE_FLAGS. Every invalidation answers for a global translation: - `Owed::discharge` of a leaf whose prior entry was global is INVLPG, unconditionally: INVPCID type 0 and a CR3 write both leave a global translation, and it is held under every CR3, not only the target's. - A `Context` owed in the kernel half is `flush_tlb_all`. - `flush_tlb_all` without PCID is `control_regs::flush_global`, a PGE toggle, where it was a CR3 reload that leaves global translations. - `guard_4k` discharges each of its two writes (both priors global, so one INVLPG each) in place of its full local flush. CR4.PGE joins CR4_REQUIRED, read from CPUID.01H:EDX[13], so `self_check` asserts it on every CPU and a CPU without it is refused by name. The declaration's own write passes through PGE clear, so global translations a firmware that left PGE set may hold are dropped when the kernel takes CR4. Two departures from the design's text, both forced by the tree: - `flush_global` toggles the live CR4, not the declared one: `flush_tlb_all` runs before the BSP's declaration (`panic_console::remap`'s `map_mmio`, main.rs, before `percpu::init_bsp`), when DECLARED_CR4 is still 0. - `pat::init` already carried the same flush as a private `flush_tlb`; it now calls `flush_global`, which is then the one CPU-state write outside the declaration. The const assertion over every `Prot` the design named is not written: PAGE_GLOBAL is named in `kernel_leaf` and `Owed::discharge` alone, which a reader sees. The two claims a stale global translation would break are metal rows on the test kernel, `a_global_kernel_leafs_discharge_drops_it` and `the_machine_flush_drops_a_global_kernel_translation`, driving SYS_DEBUG action 28 (`GLOBAL_LEAF_REWRITE`): a global kernel leaf at the kernel half's last 2 MiB, read, rewritten to a second frame, invalidated as asked and read again, on one CPU with interrupts closed. Each runs its control first, the rewrite with nothing invalidated, and refuses itself if that read already sees the new frame. Metal rows, the cheaper tier, and the one that reaches every wrong flush: this host's TCG guest drops every translation on a CR3 write and has no INVPCID, so a CR3 reload or INVPCID type 3 in place of the machine flush reads fresh there. The `control_regs` metal judge and its host check name PGE as required. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
|
Mutation patches, each a checked patch against
diff --git a/kernel/src/arch/x86_64/paging.rs b/kernel/src/arch/x86_64/paging.rs
--- a/kernel/src/arch/x86_64/paging.rs
+++ b/kernel/src/arch/x86_64/paging.rs
@@ -278,7 +278,8 @@
if let Some(have) = pcid_active() {
crate::arch::cpu::invpcid(have, Invpcid::AllIncludingGlobal, 0, 0);
} else {
- crate::arch::control_regs::flush_global();
+ // SAFETY: MUTATION, never lands: the CR3 reload this kernel flushed with before.
+ unsafe { crate::arch::cpu::write_cr3(crate::arch::cpu::read_cr3()) };
}
}
diff --git a/kernel/src/arch/x86_64/cpu.rs b/kernel/src/arch/x86_64/cpu.rs
--- a/kernel/src/arch/x86_64/cpu.rs
+++ b/kernel/src/arch/x86_64/cpu.rs
@@ -246,8 +246,8 @@
Address = 0,
/// Every entry tagged with one PCID, global pages excepted.
SinglePcid = 1,
- /// Every entry in every PCID, global pages included.
- AllIncludingGlobal = 2,
+ /// MUTATION, never lands: type 3, every entry in every PCID, global pages excepted.
+ AllIncludingGlobal = 3,
}
/// Discard TLB entries by descriptor type; both instruction faults are excluded by the signature ([`Invpcid`], [`PcidActive`]).
diff --git a/kernel/src/arch/x86_64/paging.rs b/kernel/src/arch/x86_64/paging.rs
--- a/kernel/src/arch/x86_64/paging.rs
+++ b/kernel/src/arch/x86_64/paging.rs
@@ -210,8 +210,6 @@
fn discharge(self, target: Cr3) {
match self {
Self::Nothing => {}
- // `INVLPG` drops the page's global translation in every PCID.
- Self::Address { va, prior } if prior & PAGE_GLOBAL != 0 => crate::arch::cpu::invlpg(va),
Self::Address { va, .. } => {
if let Some(have) = pcid_active() {
crate::arch::cpu::invpcid(have, Invpcid::Address, target.pcid(), va);The staging script ( #!/bin/sh
# Stages every metal arm for PR #873. Each `--metal-readback` run builds its
# images and writes <dir>/request.txt; exit 2 is Verdict::Staged.
set -u
S=/private/tmp/claude-502/-Users-jan-Dev-jan-toyos/80e90c3a-7449-46b8-84cc-b59e9dd31ae9/scratchpad/pge
M=$S/metal
AMD='<user>@<amd-bench>'
HEAD=363e0e84186184cce7bf7422ecb59f0029173d02
ROWS_NEW="a_global_kernel_leafs_discharge_drops_it the_machine_flush_drops_a_global_kernel_translation"
stage() { # tree dir host rows...
tree=$1; dir=$2; host=$3; shift 3
if [ -n "$host" ]; then
(cd $tree && cargo test --test toyos-build -- --metal --metal-readback $M/$dir --metal-host "$host" "$@") > $M/$dir.stage.log 2>&1
else
(cd $tree && cargo test --test toyos-build -- --metal --metal-readback $M/$dir "$@") > $M/$dir.stage.log 2>&1
fi
rc=$?
at=$(cd $tree && git rev-parse HEAD)
echo "$dir: tree $at rows $* EXIT=$rc"
}
# base-t14 and base-amd were staged from d4c2f8ebe by this script's first run.
stage /Users/jan/Dev/jan/toyos-pge-scout scout-t14 "" scout_tlb_base scout_tlb_nopcid
stage /Users/jan/Dev/jan/toyos-pge-scout scout-amd "$AMD" scout_tlb_base tlb_shootdown_cost
stage /Users/jan/Dev/jan/toyos-pge-mut head-t14 "" $ROWS_NEW control_regs
stage /Users/jan/Dev/jan/toyos-pge-mut head-amd "$AMD" $ROWS_NEW control_regs
mutate() { # patch dir host
p=$S/mut/$1.patch; dir=$2; host=$3
cd /Users/jan/Dev/jan/toyos-pge-mut || exit 90
git reset -q --hard $HEAD
git apply --check $p; rc=$?; echo "$1: git apply --check EXIT=$rc"
git apply $p || exit 91
printf 'MUTATION, never lands: %s\n' "$1" > $S/mut/$1.msg
git commit -q -a -F $S/mut/$1.msg; echo "$1: committed locally as $(git rev-parse HEAD), never pushed"
stage /Users/jan/Dev/jan/toyos-pge-mut $dir "$host" $ROWS_NEW
git reset -q --hard $HEAD
echo "$1: restored to $(git rev-parse HEAD); porcelain:"; git status --porcelain --ignore-submodules=none; echo "(end)"
}
mutate m1-nopcid-flush-is-a-cr3-reload m1-amd "$AMD"
mutate m2-invpcid-type-3-in-the-machine-flush m2-t14 ""
mutate m3-no-global-arm-in-discharge m3-t14 ""
mutate m3-no-global-arm-in-discharge m3-amd "$AMD"
echo ALL-DONEIts output, first run (base arms): Second run: |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The ten |
|
The T14 and AMD run request ( |
Two stagings of one metal selection from one clean committed tree in one checkout came out with different sha256s during PR #873's metal runs. Six such pairs were diffed byte by byte, per GPT region, partition, FAT file and directory-entry field. Each pair differs in 391 to 413 bytes; kernel, loader, ROOT and LOG are identical in all six. Every differing byte is either a GUID that build/image.rs draws from gptwrite::random_guid on each call (the disk's, each partition's, /toyos/log.guid, the slot table's) with the CRCs over them, or the clock, which issues/an-images-signed-version-and-its-fat-dates-are-the-hosts-clock.md already holds. Nothing filed held the GUIDs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
Metal judgement at
|
| judge | rows | result (M) |
|---|---|---|
| T14 base | scout_tlb_base, scout_tlb_nopcid |
2 passed, exit 0 |
| T14 scout | scout_tlb_base, scout_tlb_nopcid |
2 passed, exit 0 |
| T14 head | both new rows, control_regs |
3 passed, exit 0 |
| T14 m3 | both new rows | discharge FAIL, flush PASS, exit 1 |
| T14 m2 | both new rows | discharge PASS, flush FAIL, exit 1 |
| AMD base, session 1 | scout_tlb_base |
1 passed, exit 0 |
| AMD base, session 2 | tlb_shootdown_cost |
1 passed, exit 0 |
| AMD scout, session 2 | scout_tlb_base, tlb_shootdown_cost |
2 passed, exit 0 |
| AMD head, m1, m3 | both new rows | no reading: each shared-debug boot panicked (below) |
The kill condition, applied as written
1. AMD, k=4: head p50 ≤ 0.98 × base p50, with disjoint min–max ranges. FAILS.
| k | base p50 [min–max] (M) | head p50 [min–max] (M) | head − base (D) | ranges (D) |
|---|---|---|---|---|
| 0 | 10446 [10167–10531] | 10645 [10455–10743] | +199 (+1.91 %) | overlap |
| 1 | 10551 [10482–10601] | 10803 [10668–10840] | +252 (+2.39 %) | disjoint, head above |
| 4 | 10610 [10525–10692] | 10823 [10741–10883] | +213 (+2.01 %) | disjoint, head above |
| 16 | 10832 [10781–10869] | 10965 [10914–11014] | +133 (+1.23 %) | disjoint, head above |
| 64 | 11573 [11401–11617] | 11745 [11563–11856] | +172 (+1.49 %) | overlap |
- The bound is 0.98 × 10610 = 10397.8 (D). The head reads 10823 (M).
- Both arms are what they claim to be. Every CPU reads
pge=1. Base holdscr4=0x00300f68; head holdscr4=0x00300fe8, which has bit 7 set (M). - The AMD pair did not run in one session, as the design asked. Base ran in the first session and head in the second. Here is what that means:
- issues: the kernel performance audit's findings, priced by three scouts, and the speed track restaged by them #866's scout ran the same base commit
d4c2f8ebeand read 10806 at k=4 (M, comment). That is 1.8 % from this run's 10610 (D). - So the spread between sessions is about the size of the head's excess. This run does not prove the head is 2 % slower.
- It does prove the head has no 2 % gain. Against issues: the kernel performance audit's findings, priced by three scouts, and the speed track restaged by them #866's base reading the bound is 0.98 × 10806 = 10589.9 (D), and the head's 10823 is still above it.
- So the kill does not rest on the session split.
- issues: the kernel performance audit's findings, priced by three scouts, and the speed track restaged by them #866's scout ran the same base commit
2. T14, scout_tlb_base, k = 1, 4, 16, 64: no head p50 above base's with disjoint ranges. HOLDS.
Readbacks: base, head. Both ran in one session.
| k | base p50 [min–max] (M) | head p50 [min–max] (M) | head − base (D) | ranges (D) |
|---|---|---|---|---|
| 1 | 16359 [16343–16583] | 16306 [16275–16561] | −53 (−0.32 %) | overlap |
| 4 | 16370 [16331–16551] | 16416 [16315–16731] | +46 (+0.28 %) | overlap |
| 16 | 16428 [16408–16683] | 16409 [16361–16671] | −19 (−0.12 %) | overlap |
| 64 | 16713 [16678–17096] | 16668 [16636–16821] | −45 (−0.27 %) | overlap |
- Only one head p50 is above base's, at k=4, and its ranges overlap.
- Base holds
cr4=0x00320f68and head0x00320fe8on all 8 CPUs (M).
Reported, not gated
-
AMD full walks per round trip at k=0 fell from 79.10 to 26.04 (D, −67 %). issues: the kernel performance audit's findings, priced by three scouts, and the speed track restaged by them #866 defines a full walk as an L2 dTLB or L2 iTLB miss, summed over both sides. The design predicted under 40.
- The per-side counters at k=0 (M) are base parent
["29.46", "28.88", "0.00", "6.00"]and child["34.82", "34.11", "0.00", "10.11"], and head parent["3.86", "3.86", "0.00", "5.05"]and child["9.12", "9.12", "0.00", "8.01"]. - Split by kind (D): L2 dTLB misses went from 62.99 to 12.98 and L2 iTLB misses from 16.11 to 13.06.
- At k=4, full walks went from 87.21 to 34.14 (D).
- The per-side counters at k=0 (M) are base parent
-
T14 iTLB walks per round trip, parent / child (M):
- Base reads 4.00 / 4.00 at k=1 and 4.01 / 4.00 at k=4. Head reads 3.00 / 2.02 at both.
- At k=64 the parent does not fall: 4.02 → 4.01.
- issues: the kernel performance audit's findings, priced by three scouts, and the speed track restaged by them #866's run of the same base commit read 4.00 / 2.02. So the child's base figure moved between boots, and this run alone does not establish that the child's count fell.
-
T14 nopcid arm, head against base. Readbacks: base, head.
- The gain (D): −578 (−3.05 %) at k=0, −521 (−3.08 %) at k=1, −566 (−3.34 %) at k=4, −512 (−2.99 %) at k=16, and −472 (−2.59 %) at k=64.
- The ranges are disjoint at k=1, 4 and 64. They overlap at k=0 and at k=16, where the head's max is 18045.
- At k=4, walks per round trip (the first three counters, both sides) fell from 80.26 to 43.46 (D).
-
AMD
tlb: bench(latencycase): one boot per arm, both in session 2, each 256 shootdowns across 8 CPUs. Readbacks: base, head.arm min (M) p50 (M) p90 (M) p99 (M) max (M) base 1032 1342 1823 2895 13957 head 1272 1493 1673 3516 12304 All figures are in ns. The p50 rose by 151 ns (+11.3 %) and the p99 by 621 ns (+21.5 %), while the p90 fell by 150 ns (D).
What these readings say together, for the owner.
- The design said a gain under 2 % would mean "the kernel's entries are a minority of what each flush throws away". The counters say otherwise.
- On the AMD at k=0, the head removes 53 of the 79 full walks per round trip (D). So the kernel's translations were most of them, yet the round trip did not get cheaper.
- On the T14 with PCID off, which flushes on every CR3 write as the AMD does, the same change saves about 3 %.
- So what fails is the premise that removing those walks buys back cycles, and it fails on Zen 2 alone. This run does not measure why.
The correctness rows
T14, from the judges (M):
| boot | a_global_kernel_leafs_discharge_drops_it |
the_machine_flush_drops_a_global_kernel_translation |
against the request |
|---|---|---|---|
| head | PASS | PASS | as expected |
m3: Owed::Address global arm removed |
FAIL, exit 101: "the write's discharge left the old frame's global translation" (log) | PASS | as expected |
m2: INVPCID type 2 → 3 |
PASS | FAIL, exit 101: "the shootdown's flush left the old frame's global translation" (log) | as expected |
control_regspasses: 8 of 8 CPUs holdcr4=0x00320ee8, which has bit 7 set (M, log).- No boot printed the control's refusal (M), so the T14 held the translation every time. Its mutation evidence stands.
AMD bench.
- The head, m1 and m3
shared-debugboots each panicked at boot and came back without aBoot: completerecord. The harness refused all three (M). - The panic is at
kernel/src/arch/x86_64/amdvi/mod.rs:325:9: "amdvi: the unit's requesters overflow the actuator's 256-entry ring" (head's flash log). - x86-64 kernel leaves are global under a declared CR4.PGE, and only a kernel-half change drops them #873 has nothing to do with it. It is a defect already on
main, in theiommu-firmware-leftboot actuator's 256-entry ring, whichshared-debug's selftest parameters arm. x86-64 kernel leaves are global under a declared CR4.PGE, and only a kernel-half change drops them #873's diff touches noamdvifile. Its fix is being designed separately. - The head
testcasesboot booted, and the harness's boot verdict ispassed(M).control_regswas not judged as a row there. Its log reads 8 of 8 CPUs holdingcr4=0x00300ee8, which has bit 7 set (M, log).
What the panic leaves unproven
- m1 has no reading at all, and m1 is AMD-only.
- It reverts
flush_tlb_all's non-PCID arm to a CR3 reload. That arm runs only where PCID is off, so the T14'sshared-debugboot never reaches it. - So on no bench has a metal run shown either of two things: that the flush row sees a CR3 reload there, or that
flush_global'sPGEtoggle drops a global translation. - That toggle is the path every shootdown takes on the AMD.
- It reverts
- Neither new row has run on Zen 2. There is no head green and no m3 red there.
- Whether the control reads stale on Zen 2 is still open. It is one of the body's open questions.
- None of this moves the verdict, which reads only the scout rows, and those ran on both benches. These gaps would bar a landing if the owner overrode the kill.
Verdict
Killed. Condition 1 fails against both readings of the base, and condition 2 holds. The readings go to the owner. issues/page-global-is-a-decision-nobody-has-made.md stays.
|
Owner's ruling, recorded by the orchestrator. He was shown the kill reading (#873 (comment)) and what the published record says:
He ruled: "if the global cache is still considered best practice then we should do it else if intel and amd go away from it we leave it. we listen to what the manufacturers and reports say." Neither vendor nor the record has moved away from it. So the kill condition is overridden and the build proceeds. The AMD row that killed it is a TLB-shootdown round trip, the case where the global flush's cost lands. And base and head ran in two sessions, 1.8% apart on the same base commit. Before landing:
|
|
Review of #873 at Size:
Judged against the SDM and the APM, no finding
BLOCKER
NOTE
OWED before landing (per the brief, not counted against the code)
SEND BACK |
|
Correction to the ruling record, by the orchestrator. In #873 (comment) I wrote that "the AMD row that killed it is a TLB-shootdown round trip". That is false. The row that failed condition 1 is Design acceptance for the fix round, under the owner's ruling to follow the manufacturers' documents (review #873 (comment)):
|
…e published state of the art back The owner's answers to round 1's questions on #885: - "Regresses nothing" was never his, and goes. A design that its hardware's manufacturers or the published state of the art back follows and cites them. Bench numbers on it are reported and never block it, and a slowdown seen is investigated, not used to reject it. - A design nothing published covers is reviewed on its correctness and simplicity. A speed claim is measured and reported but does not decide by itself, unless the design's only purpose is speed. - What moves a design off its manufacturers' line is the manufacturers and the published record moving: "if intel and amd go away from it we leave it" (#873). The review's notes: the list of vendor documents goes, since "that hardware's manufacturers" already names them and the makers of every device too; the two orchestrator clauses that restated implementer.md go; and reviewer.md's Evidence sentence is main's again, since the reading it asks for is still owed and only its weight changed, which Fit now says. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
…MU_ALL The actuator wrote one INVALIDATE_DEVTAB_ENTRY for every requester id its unit's IVHD names into a 256-entry ring it never refilled, and asserted once 255 were written. The AMD bench's IVHD names 65,535 ids, so every shared-debug boot there panicked at amdvi/mod.rs:325 before the unit was switched off (#873's run). 0d783ac brought the IVHD walk in (#837); QEMU's IVHD names eight functions, so the guest test never reached the bound. It now sends INVALIDATE_IOMMU_ALL (48882 Table 41: opcode 08h, every other bit reserved) and the existing COMPLETION_WAIT. Per §2.4.8 every cache is empty once it completes, which also drops translations tagged with the BLOCKED entries' DomainID 1 that a per-entry walk left behind (§2.4.2). Linux does the same in amd_iommu_flush_all_caches whenever FEATURE_IA is set. A unit whose EFR IASup (bit 6) is clear is refused by name before anything is written: both units in reach set it, the bench's 0x206d73ef22254ade and QEMU's 0x29d3. The walk, its expect on the device list, the ring bound and leave_on's ivrs and unit parameters go. QEMU 11.1.1's amdvi_inval_all never re-reads a device table entry's mode, so under QEMU the actuator's entries block nothing: amdvi_firmware_left drops its stick-never-binds claim, keeps the register half's negative control, and the gap is filed with its exit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
…s the kernel's global translations The fix round on #873 after its first review (SEND BACK) and the owner's ruling to follow the manufacturers' documents. Intel's ADL063 (682436-040) and RPL043 (740518-031), "INVLPG May Invalidate Global TLB Entries Only For The Current PCID": with PCIDs enabled, INVLPG may leave a global translation cached for another PCID. Every kernel-half discharge of this branch is an INVLPG of a global translation, so on such a CPU below the microcode that fixed it, `toyos-cpuvuln::invlpg_misses_global` answers yes and `control_regs` withholds PCIDE, logging why. The rows are the signatures each document's Table 2-1 names beside its erratum: every Alder Lake and Raptor Lake signature but the 6P+0E Raptor Lake die (0xB06F5), which RPL043's summary marks N/A, plus Alder Lake-N (0xB06E0), which its own specification update (764616-004) does not list. The fixed revisions are Intel's, posted to linux-kernel as Message-ID 20240325231300.qrltbzf6twm43ftb@desk; no specification update carries one. The erratum does not apply in VMX non-root operation, so a guest keeps PCID, and a revision is compared signed (SDM Vol. 3A Table 12-7). The kernel reads the revision on an Intel CPU alone, by §12.11.7.1's procedure (`cpu::intel_microcode_revision`). A shootdown now names the half it changed (`invalidation::Half`). The protocol (`shootdown.rs`) counts wide generations before issuing them, and a serve answers wide where a wide generation was issued since its last wide answer, so a coalesced answer reaches as far as the widest generation it covers. x86-64 maps a wide answer to `flush_tlb_all` (INVPCID type 2, or the PGE toggle) and a narrow one to the new `flush_tlb_user` (INVPCID type 3, or a CR3 reload), which keeps global translations. Only `map_mmio` and the global-leaf actuator change the kernel half; `join` stays wide, since a kernel-half change made before the machine answered issued no generation. The user-half `Owed::Context` arm without PCID is a CR3 reload again. The global-leaf actuator runs its control and its arm in one IRQ-closed section and answers UNHELD where the control read the second frame. A third metal row, `a_user_half_shootdown_keeps_a_global_kernel_translation`, holds the user-half flush; the kernel-half row is renamed `a_kernel_half_shootdown_drops_a_global_kernel_translation`. Four loom models hold the wide reach. The track issue is deleted: this branch builds it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
… 2's gates, mutations, staging and oracle
round2/gates/ cargo run -- --ci host, cargo test (the whole guest suite), and cargo run -- --build-only on
x86-64 and AArch64, each log ending in its exit; *.head is the commit each ran at.
round2/mutations/ every mutation patch against 7ea05913b: l1-l4 (kernel-loom's tlb_shootdown) and c0-c3
(toyos-cpuvuln), run by host-mutations.sh into host-mutations.out and one log each; m1-m4,
the metal ones, staged by ../stage/stage.sh (m2, m3, m4 on the T14) and, once #884 lands, by
../stage/stage-amd-after-884.sh (m1, m3, m4 on the AMD bench).
round2/stage/ stage.sh and its output, each staged directory's own request.txt and staging log, the master
request.txt, and the AMD shared-debug script that waits for #884.
round2/oracle/ sources.md: the Intel documents the INVLPG verdict rests on, by number, revision, URL and
the sha256 of the PDF as fetched, with the lines quoted by page.
Local paths are masked: <scratchpad>, <dev>, <home>, <tmp>, <tmpdir>.
…s the kernel's global translations The fix round on #873 after its first review (SEND BACK) and the owner's ruling to follow the manufacturers' documents. Intel's ADL063 (682436-040) and RPL043 (740518-031), "INVLPG May Invalidate Global TLB Entries Only For The Current PCID": with PCIDs enabled, INVLPG may leave a global translation cached for another PCID. Every kernel-half discharge of this branch is an INVLPG of a global translation, so on such a CPU below the microcode that fixed it, `toyos-cpuvuln::invlpg_misses_global` answers yes and `control_regs` withholds PCIDE; on an Intel CPU it logs the signature and the revision it read, so a boot can be held to Linux's reading of the same CPU, and says where it withholds PCID. The rows are the signatures each document's Table 2-1 names beside its erratum: every Alder Lake and Raptor Lake signature but the 6P+0E Raptor Lake die (0xB06F5), which RPL043's summary marks N/A, plus Alder Lake-N (0xB06E0), which its own specification update (764616-004) does not list. The fixed revisions are Intel's, posted to linux-kernel as Message-ID 20240325231300.qrltbzf6twm43ftb@desk; no specification update carries one. The erratum does not apply in VMX non-root operation, so a guest keeps PCID, and a revision is compared signed (SDM Vol. 3A Table 12-7). The kernel reads the revision on an Intel CPU alone, by §12.11.7.1's procedure (`cpu::intel_microcode_revision`). A shootdown now names the half it changed (`invalidation::Half`). The protocol (`shootdown.rs`) counts wide generations before issuing them, and a serve answers wide where a wide generation was issued since its last wide answer, so a coalesced answer reaches as far as the widest generation it covers. x86-64 maps a wide answer to `flush_tlb_all` (INVPCID type 2, or the PGE toggle) and a narrow one to the new `flush_tlb_user` (INVPCID type 3, or a CR3 reload), which keeps global translations. Only `map_mmio` and the global-leaf actuator change the kernel half; `join` stays wide, since a kernel-half change made before the machine answered issued no generation. The user-half `Owed::Context` arm without PCID is a CR3 reload again. The global-leaf actuator runs its control and its arm in one IRQ-closed section and answers UNHELD where the control read the second frame. A third metal row, `a_user_half_shootdown_keeps_a_global_kernel_translation`, holds the user-half flush; the kernel-half row is renamed `a_kernel_half_shootdown_drops_a_global_kernel_translation`. Four loom models hold the wide reach. The track issue is deleted: this branch builds it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
|
Hand-off, by the orchestrator. The microcode loader (#886) lands after this branch. It will thread its |
… 2 again, at the amended head
The round's commit was amended (7ea05913b -> 86877feb6) so that control_regs logs, on an Intel CPU,
the signature and the microcode revision it read. Every gate, every host mutation and every staged
arm the head decides ran again there.
round2/gates/ cargo run -- --ci host, cargo test (the whole guest suite), and cargo run -- --build-only on
x86-64 and AArch64, at 86877feb6 (gates.head); each log ends in its exit.
round2/mutations/ host-mutations.out and the eight logs: l1-l4 and c0-c3 at 86877feb6. host-mutations-1.out is
the same script's run at 7ea05913b. The patches are unchanged: neither touches control_regs.rs.
round2/stage/ stage.sh as it ran the second time; stage-1.out, its first run at 7ea05913b, which staged the
two base arms from d4c2f8ebe (unmoved since); stage-2.out, every other arm at 86877feb6 and
the scout tree at 0bc71d325; each directory's request.txt and staging log; the master request.
round2/oracle/ unchanged.
Local paths are masked: <scratchpad>, <dev>, <home>, <tmp>, <tmpdir>.
|
The AMD bench's missing boot, read from its own journal by the orchestrator. With the readback analysis (the kernel completed, the black box read DONE, and loader pass 2 ran its The head boot is being re-run for a readback the judge can read. |
#884 fixes the `iommu-firmware-left` ring whose overflow panicked every shared-debug boot on the AMD bench, which this branch's global-leaf rows need. #871 declares XCR0, IA32_XSS and CR4.OSXSAVE in the same control_regs declaration this branch adds CR4.PGE to. Four conflicts, each both sides kept: - kernel/cpuvuln/Cargo.toml, src/lib.rs: main's erratum_1386 (AMD's revision guides) beside this branch's invlpg_misses_global (Intel's specification updates), in the description and the crate doc. - kernel/src/arch/x86_64/control_regs.rs: the header names main's XCR0, IA32_XSS and save form, and keeps this branch's one other CR4 writer, flush_global; CR4_REQUIRED carries both PGE and OSXSAVE. supported() merged cleanly: this branch's CPUID.01H:EDX[13] row beside main's CPUID.01H:ECX[26] one. init() writes CR4 through PGE clear, then EFER, XCR0 and IA32_XSS. - tests/checks.rs: main widened each case to (cr0, cr4, xcr0) and moved DECLARED's cr4 to 0x0034_0668 (OSXSAVE); this branch adds bit 7, so 0x0034_06e8, swaps the never-named example from PGE to PKE, and adds "every CPU without PGE": 15 refusals. Clean merges checked by reading: dispatch.rs and syscall.rs (action 28 is still free), tests/toyos.rs (main's control_regs judge names OSXSAVE and XCR0 and not PKE; this branch's PGE row and three metal rows sit beside main's), cpu.rs (main's leaf_d, xgetbv0, xsetbv0 beside this branch's revision read and INVPCID type 3), and idt/tlb.rs, whose entry #871 rewrote and which still calls the same serve_ipi. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
… 3, the merge with main at ab655a115 and the AMD shared-debug arms round3/gates: --ci host, --build-only on x86-64 and AArch64, and the whole guest suite at 854509da1, each log ending in its exit and the host load before it; gates.sh and its output; gates.head names the head. round3/stage: stage.sh and its two runs' output (stage-mut.out: m1, m3, m4 for the AMD bench from toyos-pge-mut; stage-head.out: the head's arms for the AMD bench and the T14), each arm's staging log and the harness's own request, every image's sha256 (shas.txt), and request.txt, which orders the boots and states each verdict expected. round3/mutations: the three patches, each git apply --check exit 0 at 854509da1. round3/images: elfs.py and its output: which embedded ELF each shared-debug image carries. Masked: home, scratchpad and temporary paths.
|
Review of #873 at Size: Round 1's BLOCKER
The metal evidence, read from the judge logs
Reliance on Linux (the owner's direction)
BLOCKER
NOTE
SEND BACK |
|
Ruling after round 3 (#873 (comment)): option (b). Intel's own workaround for ADL063/RPL043 is "INVPCID type 2 instead of INVLPG". Wherever BLOCKER 3 stands: every metal reading goes in the body with its command, exit and log, published on |
…wt/toyos-pge No conflict. Of the files both sides changed, only tests/toyos.rs: main drops proctest and the launch_authority row; this branch adds three RUST_SKIP entries, three metal rows and PGE in the control_regs judge, each in a hunk main does not touch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
…nd, and PCID stays on every CPU that has INVPCID Intel's specification updates for Alder Lake (682436, ADL063) and Raptor Lake (740518, RPL043) say INVLPG "may fail to invalidate TLB entries for global pages with PCIDs different from the current PCID value", and give the software workaround: "this can be worked around by software using INVPCID type 2 instead of INVLPG". Round 2 instead withheld PCIDE on the CPUs those documents name, below a fixed microcode revision; the owner's ruling on #873 takes Intel's workaround, everywhere PCID is on. - paging: Owed::discharge's global arm is INVPCID type 2 wherever PCIDE is set, and INVLPG only without it, where the erratum does not apply (it "applies only when PCIDs are enabled"). Every other global discharge already went through flush_tlb_all, which is type 2 under PCID. Its one caller today is guard_4k, at boot. - The verdict goes whole: toyos-cpuvuln's invlpg.rs, its table rows and description; cpu::intel_microcode_revision; control_regs' per-CPU signature and microcode line; the seven host tests. - control_regs: a CPU with PCID but no INVPCID gets no PCIDE, as before, and now says so on the log by name, since every global discharge under PCID needs INVPCID. The review's two NOTEs: - the_t14_keeps_pcid went with the verdict. - The keeping row's action refuses, answering GLOBAL_LEAF_WIDENED, when this CPU's own answers inside the user-half shootdown reached wide because another CPU's kernel-half generation landed in it. Shootdown::widened, test-actuators only, reads the per-CPU record the protocol already keeps; kernel-loom declares the feature for cfg checking. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
… 4, and every metal reading of rounds 2 and 3
x86-64's kernel leaves are now global under a declared
CR4.PGE, as AArch64's already are. Every kernel-half leaf gets the bit through one constructor,kernel_leaf. A kernel-half change drops global translations: its discharge isINVPCIDtype 2 whereverPCIDEis set andINVLPGwhere it is not, and its shootdownINVPCIDtype 2 or aPGEtoggle. A user-half change keeps them: its shootdown isINVPCIDtype 3 or aCR3reload. A CPU with PCID but noINVPCIDis declared withoutPCIDE, and the log names that refusal. This buildsissues/page-global-is-a-decision-nobody-has-made.md, which it deletes.Head
806130ce9: round 1's363e0e841, round 2's86877feb6, round 3's merge854509da1, the merge withmainat1a7464692(cca8d6fb6, no conflict), and this round's806130ce9. A draft. Still owed before it lands: this round's metal run (requested below) and a review.The record
ToyOSOrg/evidence@ffa5669). Condition 1 was the AMD bench'sscout_tlb_baseat k=4: head p50 10823 against base 10610, +2.01 %, against a bound of 0.98 × base. Condition 2 was the T14'sscout_tlb_baseat k = 1, 4, 16, 64: no head p50 was above base's with disjoint ranges.PCIDEon the CPUs the erratum names.INVPCIDtype 2, not losing PCID. The ruling (option (b)) is that wherever PCID is on, every global discharge usesINVPCIDtype 2. The erratum verdict is deleted whole. A CPU with PCID but noINVPCIDgets noPCIDE, refused by name. The authority is Intel's specification updates and the SDM. This round builds that ruling.What changed, per decision
1. Which leaves are global (
kernel/src/arch/x86_64/paging.rs)PAGE_GLOBAL(bit 8) is set in one place,kernel_leaf(cache). It sits besideProt::leaf_bits, which builds the user leaves and never names the bit.map_2m, the direct map andmap_mmio, go throughkernel_leaf.guard_4kcopies the 2 MiB leaf's flags into its 4 KiB PTEs.2. A kernel-half change drops global translations
Owed::discharge,Addresswith a prior global leafINVPCIDtype 0 on the target PCID; without PCID,INVLPGonly wheretargetis loadedPCIDE,INVPCIDtype 2; without it,INVLPG(va)Owed::discharge,Contextin the kernel halfCR3reloadflush_tlb_all; nothing reaches it todayflush_tlb_allINVPCIDtype 2, or aCR3reloadINVPCIDtype 2, orcontrol_regs::flush_global, thePGEtoggleguard_4ksubsumed_by_flush, then a full local flush3. A user-half change keeps them
arch::tlb::shootdown(origin, half), withinvalidation::Half. Onlymap_mmioand the global-leaf actuator change the kernel half.kernel/src/shootdown.rs):issue(Reach::Wide)counts the wide generation before issuing it (Release).Acquire). It answers wide where the count has passed the one its last wide answer read, and records the new count after the flush. So one answer that covers both a narrow and a wide generation is wide.flush_tlb_alland a narrow one to the newflush_tlb_user:INVPCIDtype 3, or aCR3reload. Both keep global translations (SDM Vol. 3A §4.10.4.1, APM Vol. 2 §5.5.2).joinstays wide: a kernel-half change made before the machine answered shootdowns issued no generation to count.4. Intel's INVLPG erratum:
INVPCIDtype 2 under PCID (this round)ADL063 (682436-040) and RPL043 (740518-031) read: "INVLPG may fail to invalidate TLB entries for global pages with PCIDs different from the current PCID value." The scope is: "It applies only when PCIDs are enabled". The workaround is: "this can be worked around by software using INVPCID type 2 instead of INVLPG" (sources).
Owed::discharge's global arm isINVPCIDtype 2 whereverPCIDEis set, on every CPU, not only on the steppings Intel names. WithoutPCIDEit isINVLPG, which the erratum's scope leaves out. Every other global discharge already went throughflush_tlb_all, which is type 2 under PCID. So withPCIDEset, noINVLPGis left on a global translation.INVLPG. Today that meansguard_4k's two writes, at boot alone. PCID stays on every CPU that hasINVPCID.control_regs::supportedrefusesPCIDEto a CPU with PCID but noINVPCID, as before, because withPCIDEset every global discharge needs type 2. The refusal is now logged by name:control_regs: cpuN has PCID but no INVPCID: PCIDE refused.toyos-cpuvuln'sinvlpg.rs, itstable.rsrows and itsdescriptionclause (the crate is back tomain's, byte for byte);cpu::intel_microcode_revision;signature … microcode …log line;This closes round 3's second BLOCKER (the sibling of
toyos_microcode::Revision) and removes the mailing-list dependency. The design cites no other implementation.5. The CPU-state declaration (
kernel/src/arch/x86_64/control_regs.rs)cr4::PGEis inCR4_REQUIRED, fromCPUID.01H:EDX[13], andself_checkasserts it on every CPU.initwritesdeclared & !PGE, thendeclared. So a firmware that leftPGEset leaves no global translation behind.flush_globalreads the liveCR4, not the declaration, becauseflush_tlb_allruns before the BSP declares anything (panic_console::remap→map_mmio). It ends on the value it read, and the module header names it as the one other writer.pat::init's private copy of the same flush is gone.control_regsmetal judge and its host check (tests/checks.rs) namePGEas required. The declaredcr4in the check is User FP/vector state is saved only at a switch, the preempt count drops lock, and the build refuses stray vector instructions #871's0x0034_0668plus bit 7, and the verdict refuses 15 machines.6. The tests: three metal rows and one
SYS_DEBUGactionSYS_DEBUGaction 28 isGLOBAL_LEAF_REWRITE(sdk/abi;paging::debug_global_leaf,test-actuatorsonly; AArch64 answersNotSupported). It runs under the kernel lock, with interrupts closed, in one section:kernel_leaf2 MiB global leaf at the kernel half's last 2 MiB, over frame A, and reads through it.It answers which frame that third read saw, or it refuses:
UNHELDwhere the control already saw B;WIDENED(this round; round 3's second NOTE) where the user-half shootdown's answer on this CPU reached wide. That happens when another CPU's kernel-half generation (map_mmio) landed inside it. The actuator readsShootdown::widened, the per-CPU record the protocol already keeps, before and after itsshootdown(tlb::debug_shootdown_answered_wide,test-actuatorsonly). The guest asserts against each refusal with its own message, so a race reds the row as a refusal that names itself, never as a claim about the flush.a_global_kernel_leafs_discharge_drops_ittest_rs_global_leaf_dischargea_kernel_half_shootdown_drops_a_global_kernel_translationtest_rs_global_leaf_flusha_user_half_shootdown_keeps_a_global_kernel_translationtest_rs_global_leaf_user_flushMetal rows, not guest tests. A metal row is the cheaper tier (root
CLAUDE.md), and a guest on this host could not judge these rows:qemu64has noINVPCID(build/arch.rs).CR3write. That is my reading ofcpu_x86_update_cr3; I did not run it here.The kernel-half shootdown row would read B there whatever the flush, and the keeping row would read B and go red. All three rows are on
RUST_SKIP.Round 3's first NOTE,
the_t14_keeps_pcid, went with the verdict.Checks (root
CLAUDE.md)Negative controls.
INVPCIDtype 0 on the target PCID again. It is staged for both benches at806130ce9(below). At86877feb6the same deletion reddened exactly the discharge row on the T14, and at854509da1on the AMD bench (table below).INVLPGunder PCID stays green on every bench. Intel's text is the oracle there.kernel-loom --test tlb_shootdown. This round adds only thetest-actuatorsaccessor toshootdown.rs.Independent oracles.
Gates at
806130ce9(gates.head, script, exits)cargo run -- --ci host: clippy over the actuator kernels on both architectures; the loom models and every model control;toyos-cpuvuln;control_regs_verdictcargo run -- --build-onlycargo run -- --build-only --arch aarch64cargo test, the whole guest suite: 63 of 63, on x86-64 and AArch64virt--metal-readbackstagings at806130ce9(script, output)Verdict::Staged)Metal
Each judge is
cargo test --test toyos-build -- --metal --metal-readback <dir> <rows>from the tree named, with--metal-host '<user>@<amd-bench>'on the AMD bench. Each boot iscargo run -p toyos-metal -- --image <dir>/<boot>/image.img --readback <dir>/<boot> --fat32-check. Every readback, judge log and runner log of rounds 2 and 3 is atevidence@2878de5, masked (index). Every judge's exit is the runner's own line: T14 r2, T14 r2 base re-run, AMD r2, AMD r2 head re-run, T14 r3 (L1–3; the rest is #874's), and AMD r3 (L1–9).Round 1, at
363e0e841: the reading, every log atevidence@ffa5669.Round 2:
86877feb6(head, m2–m4); based4c2f8ebe(mainat177e72a9ewith the scout's two commits); head-with-scout0bc71d325; request.control_regscr4=0x00320ee8,PGEandPCIDEflush_tlb_all's type 2 → 3left: 0(kernel L424)left: 0(kernel L419)flush_tlb_allagainleft: 1(kernel L429)scout_tlb_base,tlb_shootdown_costTwo boots were re-run. The T14 base
latencycaseboot exited rc=2 and was re-run alone. The AMD headlatencycaseboot exited rc=1, which left its judge skipped, and was re-run alone in a later session. Each judge ran after its re-run.Round 2's figures, reported and not gated (derivation, output):
scout_tlb_baseround trip, cycles, p50tlb_shootdown_cost(tlb: bench, 256 user-half shootdowns, p50 / p99):Round 3, at
854509da1(request):control_regscr4=0x00360ee8control_regscr4=0x00340ee8,PGEand no PCIDCR3reloadleft: 0(kernel L359)left: 0(kernel L354)left: 1(kernel L364)CR3reload leaves it. The head's green shows thatflush_global'sPGEtoggle drops it.854509da1, the merge withmaintouched none of the kernel, ABI or global-leaf files, and this round changedpaging.rs's global discharge arm and the actuator,tlb.rs(the actuator only),shootdown.rs(the actuator only),control_regs.rsandcpu.rs.flush_tlb_allandflush_global, are unchanged, and so are their readings.Round 4's request (request, at
806130ce9) has eight boots: four on each bench.shared-debug(the three rows) andtestcases(control_regs).INVLPGon the AMD bench.Size
git diff --shortstat origin/main...806130ce9: 27 files, +670 −187 (numstat by file). This round's commit alone:git diff --shortstat cca8d6fb6 806130ce9, 12 files, +71 −241.kernel_leaf, theOwedarms,flush_globalabsorbingpat.rs's flush, the declaration's two-step write, the protocol's reach,flush_tlb_user,INVPCIDtype 3,Halfat every call site, and the namedINVPCIDrefusal.test-actuatorsonly: +136. That coversdebug_global_leafand itsSCRATCH,debug_shootdown_answered_wide,Shootdown::widened, the AArch64 refusal, the dispatch arm and the ABI's constants.RUST_SKIPentries, thecontrol_regsjudge and its check, andkernel-loom's check-cfg namingtest-actuators.What I am unsure of
WIDENEDnames the race instead of claiming a defect, but the metal harness has no verdict between pass and fail, so a run wheremap_mmiolands inside the keeping row's section is red. Past boot onlymap_mmioissues a kernel-half shootdown. No boot of rounds 1–3 hit this race.Shootdown::widenedhas no negative control. A version that always answered "not wide" would pass every row unless the race happened in that boot. A reader can check it: it is oneRelaxedload of the recordservewrites.CR4.PGEat handoff, and whether AMD treats bit 8 of an upper-level entry as reserved.🤖 Generated with Claude Code
https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy