Repository navigation
AMD-Vi stage 1: the kernel decodes the IVRS and switches off every AMD-Vi unit firmware left on - #837
Conversation
toyos-acpi decodes the IVRS (AMD 48882 §5.2): every IVHD of types 10h, 11h and 40h, every IVMD, and every device entry a unit's block carries, ranges paired. A unit is answered once, at the highest IVHD type the table holds for it; lower ones are answered as superseded. Opening the table judges every block header, IVMD range, duplicate and the IVHD count; a unit's device entries are judged by the walk that reads them, so a malformed list leaves the unit's register base readable, which is all switching it off needs. Entry types of 80h and above other than F0h are refused by name: ACPICA sizes them at 4 or 8 bytes and Linux does not advance past them. The kernel's x86-64 iommu_unit becomes a dispatch: VT-d where firmware publishes a DMAR, AMD-Vi where it publishes an IVRS. kernel/src/arch/x86_64/ amdvi maps each unit's 16 KiB window, logs CONTROL, Status and EFR, and turns off CmdBufEn, EventLogEn, GALogEn and PPRLogEn (each confirmed in Status) and then IommuEn, Linux's iommu_disable order. Nothing is programmed: VT-d holds no unit there, so a domain is refused NoUnit and no interrupt is remapped, recorded in issues/an-amd-vi-machine-isolates-no-device.md. The existing iommu-firmware-left actuator stages the AMD-Vi case too, as VT-d's does in its own backend: every device table entry V=1 TV=1 Mode=4 over an empty root with IR=IW=0, the command buffer and event log running, each enumerated function's entry invalidated, as QEMU 11.1.1 switches a function to translation only on INVALIDATE_DEVTAB_ENTRY and only with dma-remap=on. Profile::MetalAmdVi puts amd-iommu,intremap=off,dma-remap=on in front of the boot stick's xHCI; amdvi_firmware_left boots it plain and with the actuator and requires the stick bound behind the unit. With the switch-off skipped, the stick's Enable Slot gets no answer and it never binds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
|
guest-amdvi-2.log, part 1 of 1, at head
|
|
negative-control.patch, part 1 of 1, at head
|
|
negative-control.log, part 1 of 1, at head
|
|
guest-suite.log, part 1 of 1, at head
|
|
clippy-2.log, part 1 of 1, at head
|
|
build-only-1.log, part 1 of 1, at head
|
|
iasl-qemu-ivrs.dsl, part 1 of 1, at head
|
|
iasl-crafted-ivrs.dsl, part 1 of 1, at head
|
|
ci-host-2.log, part 1 of 9, at head
|
|
ci-host-2.log, part 2 of 9, at head
|
|
ci-host-2.log, part 3 of 9, at head
|
|
ci-host-2.log, part 4 of 9, at head
|
|
ci-host-2.log, part 5 of 9, at head
|
|
ci-host-2.log, part 6 of 9, at head
|
|
ci-host-2.log, part 7 of 9, at head
|
|
ci-host-2.log, part 8 of 9, at head
|
|
ci-host-2.log, part 9 of 9, at head
|
|
Review of #837 at Net lines: The two departures from the plan, judged.
BLOCKER
NOTE
SEND BACK |
…osing line counts the units switched off, and the event, GA and PPR interrupt enables go off with their logs `amdvi::init` and `switch_off` no longer take the enumerated functions: the `iommu-firmware-left` actuator alone used them, so the shipping build carried a parameter and two lint suppressions for it. The actuator now invalidates the device table entry of every requester the unit's own device entries name. `present` maps and checks a unit's window; `switch_off` runs only on a unit it answered, and the closing line counts those, so a unit whose base was refused or whose STATUS reads all ones is no longer reported switched off. `FIRMWARE_LEFT` clears EventIntEn, GAIntEn and PPRIntEn too, in Linux v6.17's `iommu_disable` order, each confirmed by CONTROL reading back: inert while IommuEn is off, but nothing later should inherit them as firmware set them. The deadline-and-assert spin is one function, `spin_until`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
Clean: #826 touches tests/common/qemu.rs and tests/toyos.rs away from the AMD-Vi profile and row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
… in the AMD-Vi branch's whole suite at load 118 Recorded with the host's load, as a red seen under load is; not run again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Head
|
|
Review of #837, round 4, at Net lines (
Since Earlier BLOCKERs
Earlier NOTEs
BLOCKER
NOTE
LAND |
#846, #843, #849, #850, #840, #839, #837) into the batch: the icons' and wallpaper's digests hash with toyos-sha2-hw, and the loader's wall clock reads through its own UEFI bindings The batch's merge of main at f72d53d moved #813's wallpaper and #814's icon digest tests onto `toyos_sha2`. #833, already on main, had replaced the root package's `toyos-sha2` dependency with `toyos-sha2-hw`, so CI's merge of the two compiled no `toyos-build` lib test and both the build system's tests and clippy went red with E0432. Both tests now hash through `toyos_sha2_hw`, as every other SHA-256 the build takes does. #842's `bootloader/src/wallclock.rs` was written on the `uefi` crate, which #815 removes; it now calls `efi`'s `RuntimeServices::get_time`, whose `Time` fields are plain and whose error is the `Status` itself. `start_kernel` takes main's `wall_clock` and the batch's `SystemTable`; the batch's `armed_at` goes, as main replaced it with `wallclock::now`. Cargo.lock takes main's `ntapi`; `nonempty` goes with `gix`, which the batch removed and which was its only user (`cargo metadata --offline`). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
, #836, #839, #840, #842 through #847, #849 and #850, into consent system.toml: sshserver and shell start both toyfetch (#843) and grants. tests/common/qemu.rs: Profile carries both Desktop and MetalAmdVi (#837). tests/toyos.rs: SCREEN_TESTS carries consent_prompt beside virt_wall_clock_utc (#842) and virt_low_ecam (#840). Beyond the conflict lines: #833 replaced the build's toyos-sha2 dependency with toyos-sha2-hw, so consent_prompt digests its job with toyos_sha2_hw::sha256_digest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
…MU_ALL The actuator wrote one INVALIDATE_DEVTAB_ENTRY for every requester id its unit's IVHD names into a 256-entry ring it never refilled, and asserted once 255 were written. The AMD bench's IVHD names 65,535 ids, so every shared-debug boot there panicked at amdvi/mod.rs:325 before the unit was switched off (#873's run). 0d783ac brought the IVHD walk in (#837); QEMU's IVHD names eight functions, so the guest test never reached the bound. It now sends INVALIDATE_IOMMU_ALL (48882 Table 41: opcode 08h, every other bit reserved) and the existing COMPLETION_WAIT. Per §2.4.8 every cache is empty once it completes, which also drops translations tagged with the BLOCKED entries' DomainID 1 that a per-entry walk left behind (§2.4.2). Linux does the same in amd_iommu_flush_all_caches whenever FEATURE_IA is set. A unit whose EFR IASup (bit 6) is clear is refused by name before anything is written: both units in reach set it, the bench's 0x206d73ef22254ade and QEMU's 0x29d3. The walk, its expect on the device list, the ring bound and leave_on's ivrs and unit parameters go. QEMU 11.1.1's amdvi_inval_all never re-reads a device table entry's mode, so under QEMU the actuator's entries block nothing: amdvi_firmware_left drops its stick-never-binds claim, keeps the register half's negative control, and the gap is filed with its exit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy
Stage 1 ("amdvi-off") of the AMD laptop plan. The kernel finds every AMD-Vi unit the IVRS describes and switches off whatever firmware left on, so DMA reaches memory untranslated; the boot stick's DMA is the one that matters. No unit is programmed: an AMD-Vi machine isolates no device.
issues/an-amd-vi-machine-isolates-no-device.mdrecords this, with its exit and its holder, the AMD laptop track's stages 5 (amdvi-translate) and 6 (amdvi-intremap).Head:
ff978fe7f, with origin/main merged at26b3684b9.What changed, per decision
toyos-acpi/src/ivrs.rs: the IVRS decoder, written for stage 5 to build on. It reads:Other(type).Its rules:
blocks()answers it once, asUnit, at the highest IVHD type the table holds for it, and answers the lower ones asSuperseded.ivrs()judges every block header, every IVMD range (backwards, or past the address space), two IVHDs of one type naming one unit, and the IVHD count (at most 64, which bounds the pairwise check). A unit's device entries are judged by the walk that reads them. A malformed list therefore refuses that walk and leaves the unit's register base readable, which is all switching it off needs.ivhd_entry_lengthanswers 0 for them and does not advance. F0h outside an IVHD 40h is refused too, as Linux does.kernel/src/arch/x86_64/iommu_unit.rs:iommu_unitbecomes a dispatch. VT-d runs where firmware publishes a DMAR, AMD-Vi where it publishes an IVRS, and one line is logged where it publishes neither.domain,faultandinterruptstay VT-d's. On an AMD machine VT-d holds no unit, sodomain::createisErr(NoUnit)andinterrupt::is_armedis false. The guest test asserts this by the kernel's own line.ynmoved here so they are shared, not copied.message_unittoo. It is nowiommu_unit::interrupt, so the dispatch stays the one place that says which backend serves the machine. On an AMD-Vi machine VT-d holds no unit, sois_armedis false,remapping()isNotRemappedand a kernel driver's message is the compatibility one, as atd242c7f05.iommu_virtio_platform, whose new arm refuses a process's claim where no unit is, is green besideamdvi_firmware_leftbelow.vtd::initnow returns whether a DMAR was published, andb"IVRS"joinsACPI_TABLES.kernel/src/arch/x86_64/amdvi/mod.rs: the switch-off.presentmaps each unit's 16 KiB window. It refuses a base that is not 16 KiB-aligned or is past 52 bits, and a unit whose STATUS reads all ones. Each refusal is logged with "whatever firmware left on stays on".switch_offthen turns off, in Linux v6.17'siommu_disableorder: CmdBufEn, EventIntEn, EventLogEn, GALogEn, GAIntEn, PPRLogEn, PPRIntEn and IommuEn.spin_untilunder a 1 sTripwire.amd_iommu_types.h's 3, 29 and 14.presentanswered (round 2): "AMD-Vi isolates no device this boot: N unit(s) described, M switched off". A unit whose base was refused, or whose STATUS reads all ones, is no longer reported switched off.The negative control's staging is the existing
iommu-firmware-leftactuator, not a new loader actuator. VT-d already stages "firmware left a unit on" with this kernel actuator, in its own backend. A loader actuator would be a second mechanism for the same staging.ivrs.devices(&unit)) is invalidated, and a COMPLETION_WAIT store confirms it. QEMU's eight Selects include the xHCI's 00:03.0.amdvi::initandswitch_offtherefore no longer take the enumerated PCI functions. Neither carries a lint suppression for the actuator. The actuator is called frominit, betweenpresentandswitch_off, under#[cfg(feature = "boot-actuators")].Profile::MetalAmdViis Metal's shape (the boot stick on the xHCI, no virtio), withamd-iommu,intremap=off,dma-remap=onas the first-device.Unit::{Vtd(Iommu), AmdVi}, so VT-d's knobs cannot describe an AMD machine.kernel-irqchip=splitgoes to VT-d alone.dma-remap=onis required. Without it, QEMU 11.1.1 passes every function's DMA untranslated (hw/i386/amd_iommu.c:1165,:1310). Even with it, a function switches to translation only on anINVALIDATE_DEVTAB_ENTRYnaming it.issues/the-host-gate-aborts-with-no-message-when-a-destructor-panics-during-unwinding.mdfiles the first--ci hostabort atd242c7f05: the five crash reports' frames, the head, and the volume at 99% full. Reading shows every frame reaches aDropthateprintln!s, which panics when stderr's write fails; the trigger is unconfirmed, and the exit is a host test of those destructors under a stderr that refuses writes.This branch edits no existing issue file. The guest-suite red at
192011511stays recorded in the checks table below, with its load, for the orchestrator.Checks (high-risk: the IOMMU, a trust boundary on firmware input)
Each log is posted whole as a PR comment, and the linked comment is the evidence. Every row of the first table but the last ran at
ff978fe7f, in one script, in this order; the script printed an emptygit statusbefore the first and after the last.cargo run -- --ci host: 79=== [ci]steps, all green. Load 37.8 before, 30.4 after; the data volume 88% full before, 91% after.cargo run -- --build-only --arch x86_64cargo run -- --build-only --arch aarch64cargo run -- --clippycargo test --test toyos-build -- amdvi_firmware_left, the green arm"usb-storage: 1 device(s)" never reached the boot console, afterxHCI: Enable Slot on port 1: no answer in 2000 msandusb-storage: 0 device(s). The supervisor reports the build asff978fe7f… dirty. One script ran the green arm,git apply --check, the red arm andgit apply -R, and printed an emptygit statusbefore and after.cargo test --test toyos-build, the whole guest suite: 59 of 59 passed. The x86-64 IOMMU rows areamdvi_firmware_leftandiommu_virtio_platform(both arms, with and without VT-d); the x86-64 rows on a profile with VT-d reachmessage_unitthrough their kernel drivers' MSIs. Load 30.4 before, 26.9 after.d242c7f05:toyos_acpi::ivrs, thendevices()over every IVHD, superseded ones included. The source lays the table under an RSDP and an XSDT astoyos-acpi/tests/ivrs.rsdoes; the table stays local. Counts:ivrs()Ok; 1 unit; 2 superseded IVHDs; 0 IVMDs; 0 other blocks; 3 walks; 19 entries; 0 refusals.Earlier heads, kept as record:
cargo run -- --ci hostatd242c7f05, the first run there. It aborted mid-way (SIGABRT); filed as the abort issue above. Its rerun at that head exited 0.cargo test --test toyos-build, the whole guest suite at192011511: 54 of 55 passed. The red isvirt_off_names_the_cpus_left_on(AArch64virt):STALLED: waiting for the boot's last word, at 1-minute load 118 on 14 cores. Not run again; reported to the orchestrator.The correction records that round 1's three comments labelled "at head
dc1075300" (guest-amdvi-2, negative-control.patch and negative-control.log) were run ona09dce906plus the uncommitted change.The target's reading. Two boots of the orchestrator's images on the AMD laptop, each carrying this branch at
d242c7f05:CONTROL 0x0000220000000400,STATUS 0, and EFR equal to the IVRS's EFR image. EveryFIRMWARE_LEFTbit is clear, soswitch_offtakes no write path on that silicon and nospin_untilruns.100e61b9e): one unit at IVHD 40h at0xfdf00000, the 10h and 11h IVHDs superseded, and its device entries with no refusal. The rest of those lines, unedited, continue withamdvi unit0 handed over control=0x0000220000000400 status=0x00000000 efr=0x206d73ef22254ade(the kernel's own reading after the loader, matching the survey's),switched off control=0x0000220000000400 status=0x00000000,1 unit(s) described, 1 switched off,usb-storage: 1 device(s)and the supervisor'sbuild 100e61b9e… clean.git diff d242c7f05 100e61b9eover every file this branch changed against its base, posted whole: the image touchesboot.rs(boot-step markers, a FACS hand-off and one image-only triple-fault switch),mod.rs(onei2chidmodule line),vtd/mod.rs(EIME only under x2APIC),toyos-acpi/src/lib.rsand three test files. It leavesamdvi/mod.rs,iommu_unit.rs,ivrs.rs,actuator.rs,iommu/mod.rsandlicence.rsbyte-identical.The oracles.
iasl -d(ACPICA 20260930, built from source outside the tree, never run by the build) decodes two tables field for field as the decoder does: QEMU's IVRS and the crafted every-type table. The fields are offsets, entry lengths, used ids, handles, variety, HID/CID/UID and IVMD ranges.User-Agent, over both HTTP/2 and HTTP/1.1.actbl2.h(20260930) and Linux v6.17'sdrivers/iommu/amd/{init.c,amd_iommu_types.h}, plus iasl as above.Host tests (
toyos-acpi/tests/ivrs.rs, 13).toyos-acpi/fixtures/qemu-11.1.1-amdvi/ivrs.bin, captured off a firmware-only boot bypmemsaveand a host-side walk from the RSDP;SOURCEbeside it says how.Why a guest test. No type or host test reaches a unit's register sequence or the DMA it frees, and the T14 is an Intel machine. QEMU's
amd-iommuis the only AMD-Vi unit in reach.control=0x…1005 status=0x00000018. It must say CmdBufEn, EventLogEn and IommuEn go off, and none of the other five fields.1 unit(s) described, 1 switched off, andNoUnit.Growth
git diff --shortstat origin/main...ff978fe7f: 18 files, +1667 −74. Production (kernel,toyos-acpi/src,src): +989 −55. Tests and the fixture: +596 −19. Issues: +82.boot-actuators.message_unitre-export; the issues grow by the new abort issue and the owner line.What I am unsure of
--ci hostabort's trigger (exit 134 atd242c7f05) is unconfirmed: a full volume under a redirected stderr is what reading suggests, and the system log for that minute was not readable. It is filed, with its exit.🤖 Generated with Claude Code
https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C