issues: what the shipped OS takes from QEMU, and the track to run in any open-source hypervisor

issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md

@@ -0,0 +1,44 @@
+---
+status: open
+kind: defect
+opened: 2026-10-12
+---
+
+# A hardware-reduced x86 machine has no power-off and no power button
+
+On x86-64 the kernel powers off only through the FADT's PM1a control block
+(`kernel/src/arch/x86_64/power.rs`), with the `\_S5` sleep type the holder of
+the `acpi` claim supplies, and that claim exists only over the fixed hardware:
+`toyos_acpi::fixed_hardware` refuses a FADT with `HW_REDUCED_ACPI` set
+(`lib/acpi/src/fadt.rs`), and `acpi_mode::init`
+(`kernel/src/arch/x86_64/acpi_mode.rs`) then mints no row. So on such a machine
+acpiserver gets no claim, nothing supplies S5, a shutdown is refused with
+`no ACPI server supplied S5`, and a press of the power button reaches nobody.
+
+A hardware-reduced platform has neither PM1 block nor SMI_CMD. ACPI 6.5 gives
+it the FADT's `SLEEP_CONTROL_REG` and `SLEEP_STATUS_REG` for sleep and
+soft-off (§4.8.3.7, Sleep Control and Status Registers), and the Generic Event
+Device, `ACPI0013`, for its events, the power button's among them: an
+interrupt whose `_EVT` method the AML runs (§4.1.1.1). Every AArch64 machine
+is one, which this tree meets with PSCI (`kernel/src/arch/aarch64/power.rs`).
+These ACPI sections are as a third-party copy of ACPI 6.6 numbers them; 6.5
+itself is unread.
+
+Cloud Hypervisor's x86-64 FADT sets `HW_REDUCED_ACPI` and names the two sleep
+registers and a reset register, and PM1a blocks only under an option of its
+own, at otherwise unused ports. Its power button, which its API's
+`/vm.power-button` presses (its `docs/api.md`), is a GED whose `_EVT` notifies
+`\_SB_.PWRB`. QEMU's `microvm` is hardware-reduced too. q35,
+crosvm, VirtualBox and bhyve publish the fixed hardware.
+
+**Owner**: whoever holds
+`issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`, whose
+server holds the machine's power button and S5.
+
+**Exit**: a hardware-reduced FADT and DSDT, recorded by ToyOS from a Cloud
+Hypervisor guest with their provenance as `lib/acpi/fixtures/` records QEMU's,
+decode in a host test to the sleep control register and the GED's
+power-button method, and acpiserver's host test runs that method to a press;
+and the image booted by hand under Cloud Hypervisor on x86-64, its log on the
+evidence repository under the hypervisor's version and command line, powers
+off on `shutdown` and on a press through `/vm.power-button`.

issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md

@@ -0,0 +1,54 @@
+---
+status: open
+kind: defect
+opened: 2026-10-12
+---
+
+# A virtio function that offers no `ACCESS_PLATFORM` is handed over as translated
+
+Virtio 1.2 §6, `VIRTIO_F_ACCESS_PLATFORM`: "If this feature bit is set to 0,
+then the device has same access to memory addresses supplied to it as the
+driver has. In particular, the device will always use physical addresses
+matching addresses used by the driver ... When clear, this overrides any
+platform-specific description of whether device access is limited or
+translated in any way, e.g. whether an IOMMU may be present." And §6.1: "If
+VIRTIO_F_ACCESS_PLATFORM is not offered, then a driver MUST pass only physical
+addresses to the device."
+
+`pcidev` hands a process a function once the unit the firmware's tables put
+it behind has given it a domain (`kernel/src/pcidev/mod.rs`), and never asks
+whether a virtio function offers the bit; the holder's driver accepts it where
+offered and goes on without it (`Offer::accept`, `lib/virtio/src/pci.rs`). A
+function that does not offer it is not behind the unit at all: QEMU keeps it
+on the machine's own address space whatever the `DMAR` says
+(`tests/common/iommu.rs`). So an honest
+holder's descriptors, which carry domain addresses above all memory
+(`kernel/src/iommu/window.rs`), reach nothing, and a hostile holder's, which
+carry physical addresses, reach all of memory: the claim's isolation is void
+for exactly the functions it is promised for.
+
+The launcher and the harness hide it. Each passes `iommu_platform=on` to every
+virtio function wherever a unit exists (`build/qemu.rs`), and
+`iommu_virtio_platform` reds a profile that does not; QEMU's own default is
+off. A transitional virtio function cannot offer the bit in QEMU at all, and
+VirtualBox's virtio core offers it nowhere
+(`issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md`).
+
+The check is not the holder's, for the holder is who the domain protects the
+machine from, nor the kernel's, which the owner's target keeps free of virtio
+(`issues/every-driver-is-still-in-the-kernel.md`; `kernel/src/pcidev` and
+`kernel/src/iommu` name none today). By the orchestrator's ruling (#896,
+issuecomment-6131213771, ruling 1) it is the supervisor's, which mints the
+claim and is trusted: it reads the function's offered features before it moves
+the claim in, and on a machine whose functions are translated refuses a virtio
+function that does not offer the bit. How it reads the word is the stage's
+design.
+
+**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`.
+
+**Exit**: a guest boot under the harness's VT-d with netstack's virtio-net
+created without `iommu_platform=on` reads the supervisor's line refusing
+netstack that function for its missing `VIRTIO_F_ACCESS_PLATFORM`, and
+`netstack: no NIC on this machine`; the same boot with the check deleted
+endows netstack the function; and `git grep -i virtio kernel/src/pcidev
+kernel/src/iommu` still finds nothing.

issues/diskserver-claims-qemus-nvme-by-its-pci-id.md

@@ -0,0 +1,52 @@
+---
+status: open
+kind: defect
+opened: 2026-10-12
+---
+
+# diskserver claims QEMU's NVMe by its PCI id and no other controller
+
+Both shipped boot configs give diskserver one device, `pci:1b36:0010`
+(`images/system.toml`, `images/console.toml`): the vendor and device id of
+QEMU's own `nvme` model, under Red Hat's vendor id. A claim matches the id exactly (`pcidev::claim`,
+`kernel/src/pcidev/mod.rs`), so on any other NVMe controller the supervisor
+endows diskserver nothing, it says `no NVMe controller this row names is on
+this machine; serving no partition`, and DATA, LOG and BOOT are absent unless
+the image is on a USB stick the kernel drives. The driver is not QEMU's:
+`Controller::open` (`system/diskserver/src/nvme.rs`) takes any function whose
+class code is `01:08:02`.
+
+Read, not booted:
+
+| | NVMe id | MSI-X table |
+|---|---|---|
+| QEMU `nvme` | `1b36:0010`, or `8086:5845` with `use-intel-id=on` | BAR 0, unless `msix-exclusive-bar=on` |
+| VirtualBox | `80ee:4e56` | BAR 5 |
+| bhyve | `fb5d:0a0a` | BAR 4 |
+
+Cloud Hypervisor and crosvm offer no NVMe. Every controller on metal carries
+its maker's id.
+
+The identity every NVMe controller shares is its class code: base class 01h,
+subclass 08h, programming interface 02h, the "NVM Express I/O controller" of
+the PCI-SIG's *PCI Code and ID Assignment Specification*, which QEMU, VirtualBox
+and bhyve each set. `issues/the-lan-is-not-yet-production-grade.md` plans the
+same for NICs ("init binds drivers to hardware from a declared match table by
+vendor, device and class").
+
+Two constraints a class row meets that an id row does not: a machine with two
+NVMe controllers matches twice, which `pcidev::claim` refuses as `Ambiguous`
+today; and a machine whose NVMe holds another operating system, as the T14's
+holds Ubuntu, is handed to diskserver, which then serves only the partitions
+its table and grants name (`system/diskserver/src/main.rs`).
+
+**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`.
+
+**Exit**: a `devices` row names a PCI function by class code, which
+`toyos_abi::syscall::DeviceRequest::parse` takes and refuses malformed; both
+shipped configs name NVMe that way; and a guest boot whose controller is
+QEMU's `nvme` with `use-intel-id=on` serves DATA through diskserver, which a
+row naming `1b36:0010` instead reds. The configs the T14 boots keep their id
+row, outside this exit, until the T14's NVMe holds no Ubuntu: a class row there
+would hand diskserver the disk the metal workflow reboots through (the
+orchestrator's ruling, #896, issuecomment-6131213771, ruling 4).

issues/no-claim-gets-an-address-space-under-virtio-iommu.md

@@ -0,0 +1,70 @@
+---
+status: open
+kind: defect
+opened: 2026-10-12
+---
+
+# No claim gets an address space under virtio-iommu
+
+A process is handed a PCI function only once the function has an address
+space of its own at a unit and, on x86-64, interrupt remapping
+(`kernel/src/pcidev/mod.rs`'s header; `Refusal::NotRemapped`,
+`Refusal::Untranslated`). The units this kernel programs are VT-d (`DMAR`,
+`kernel/src/arch/x86_64/vtd/`) and SMMUv3 (`IORT`,
+`kernel/src/arch/aarch64/smmu/`). It finds AMD-Vi units (`IVRS`) and programs
+none (`kernel/src/arch/x86_64/iommu_unit.rs`), so a machine whose unit is AMD-Vi,
+VirtualBox's `--iommu amd` and QEMU's `amd-iommu` among them, refuses every
+claim until `issues/an-amd-vi-machine-isolates-no-device.md` lands.
+
+QEMU emulates VT-d, AMD-Vi and SMMUv3, and VirtualBox VT-d or AMD-Vi under
+`--iommu` on its ICH9 chipset, off by default (`VBoxManage modifyvm`). Cloud
+Hypervisor offers
+virtio-iommu instead (virtio 1.2 §5.13, device id 23, PCI `1af4:1057`): its
+documentation says it implements it "in order to provide a virtual IOMMU to its
+users" and "will not try to implement a full emulation of a physical IOMMU",
+and puts a device behind it only where the device is tagged `iommu=virtio`,
+whose "default value will always be `off`" (its `docs/iommu.md`); it describes
+the unit to the guest in a `VIOT` table, and offers an SMMUv3 only for a
+passthrough device behind a physical one (`docs/iommu.md`, "Emulated ARM
+SMMUv3"). crosvm puts only a host device passed through with
+`--vfio …,iommu=viommu` behind its virtio-iommu, never a device it emulates.
+So Cloud Hypervisor by default, and crosvm for every device it emulates, are
+the no-unit case of
+`issues/a-machine-without-an-iommu-refuses-every-claim.md`, and a device
+Cloud Hypervisor is told to put behind its virtio-iommu is refused here too.
+
+**Programming a virtio-iommu puts virtio in the kernel**, on either
+architecture. Its requests travel on a virtqueue, its `requestq` (§5.13.2),
+over the virtio PCI transport, and the unit a claim's domain lives at is the
+kernel's (`pcidev`'s header). That conflicts with the owner's target that
+"`virtio` does not appear anywhere in the kernel"
+(`issues/every-driver-is-still-in-the-kernel.md`), and where a virtio-iommu
+driver lives is the owner's to rule, for both architectures.
+
+The two architectures differ in what virtio-iommu leaves undone besides:
+
+- **AArch64.** A claimed function's message is its slot's LPI through the
+  ITS (`kernel/src/arch/aarch64/irqchip/its.rs`), whose translation by
+  DeviceID is the per-function boundary; virtio-iommu need only translate DMA.
+  Cloud Hypervisor's AArch64 machine has an ITS (its `IORT`'s ITS group).
+- **x86-64.** Virtio 1.2 defines no interrupt remapping: an MSI doorbell is a
+  reserved region the driver may not map (§5.13.6.8.1,
+  `VIRTIO_IOMMU_RESV_MEM_T_MSI`), whose writes pass untranslated. Nothing mints
+  a `Remapping` there, so a claim still ends at `NotRemapped`. Whether a claim
+  may stand under virtio-iommu with its messages unremapped is the owner's to
+  rule too: `issues/every-driver-is-still-in-the-kernel.md` records his ruling
+  that "the IOMMU lands first and completely, interrupt remapping included".
+
+bhyve offers its guest no IOMMU of any kind;
+`issues/a-machine-without-an-iommu-refuses-every-claim.md` holds that case and
+the owner's ruling on it.
+
+**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`,
+blocked on the owner's rulings above.
+
+**Exit**: the owner's ruling on where a virtio-iommu driver lives, and on
+x86-64 his ruling on unremapped messages, recorded here; and on each
+architecture a claim under QEMU's `virtio-iommu-pci` (on AArch64 under `virt`
+with its emulated ITS) handed over in a domain of its own, a DMA outside its
+grants a `DMA FAULT` record, or refused by a `Refusal` that names
+virtio-iommu, as he rules.

issues/the-ap-trampoline-is-written-at-0x8000-whatever-the-memory-map-says-is-there.md

@@ -0,0 +1,35 @@
+---
+status: open
+kind: defect
+opened: 2026-10-12
+---
+
+# The AP trampoline is written at 0x8000 whatever the memory map says is there
+
+`kernel/src/arch/x86_64/smp.rs` copies the application processors' start-up
+code to physical page `0x8000` (`TRAMPOLINE_PAGE`) and starts every AP there.
+`arch::boot::reserved` withholds the page from the allocator, and
+`kernel_main` (`kernel/src/main.rs`) checks every region it withholds against
+the firmware's map except this one: "The architecture's own page is not a
+loader allocation, so it is named here". Nothing asks whether the page is
+memory at all, nor whose.
+
+It holds on QEMU's q35, whose edk2 hands over `0x0..0x87000` as conventional
+memory (the map recorded in `lib/bootmap/tests/direct_map.rs`), and the T14
+starts its CPUs through it; how the T14's map types the page was not read. It
+is a fixed address chosen for one layout: a firmware that keeps anything of
+its own at `0x8000` — runtime code, ACPI NVS, a reserved range — has it
+overwritten when the first AP starts, though the map the kernel was handed
+would have said so. No other hypervisor's map was read for it.
+
+What the architecture needs is a page below 1 MiB, its number the `STARTUP`
+IPI's vector (Intel SDM Vol. 3A, "MP Initialization Protocol Algorithm"), and
+UEFI lets the loader ask for exactly that before `ExitBootServices`:
+`AllocatePages` with `AllocateMaxAddress` (UEFI 2.10 §7.2), the page then
+`LoaderData` like every other region `kernel_main` checks.
+
+**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`.
+
+**Exit**: the trampoline's page is one the loader allocated below 1 MiB and
+hands over in `KernelArgs`, held by the check `kernel_main` makes of every
+other withheld region; deleting the allocation reds that check at boot.

issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md

@@ -0,0 +1,50 @@
+---
+status: open
+kind: defect
+opened: 2026-10-12
+---
+
+# The virtio drivers match only a non-transitional device id
+
+Virtio 1.2 §4.1.2.2: "Drivers for device types listed in section 4.1.2 MUST
+match devices with the PCI Vendor ID 0x1AF4 and the Transitional PCI Device ID
+indicated in section 4.1.2" — `0x1000` for a network card, `0x1003` for a
+console, among others — beside the `0x1040`-based id. A transitional device
+also publishes the virtio PCI capabilities (§4.1.4), and §4.1.5.1.1.1 has a
+non-transitional driver look for them and "fail gracefully" where they are
+absent. That is not the legacy interface, which this tree refuses.
+
+The tree matches the `0x1040`-based id alone:
+
+- netstack's `CARDS` (`system/netstack/src/main.rs`) and its row in
+  `images/system.toml`, `pci:1af4:1041`;
+- the kernel's console, `VIRTIO_CONSOLE_DEVICE` `0x1043`
+  (`kernel/src/drivers/virtio_console.rs`).
+
+The launcher and the harness hide it: each asks QEMU for
+`virtio-net-pci-non-transitional` and `virtio-serial-pci-non-transitional`
+(`build/qemu.rs`, `tests/common/qemu.rs`). QEMU's own default is transitional
+on q35's root bus — `disable-legacy=auto` resolves to off for a device not
+behind a PCIe port — and VirtualBox's virtio-net is `1af4:1000`. On either,
+netstack has no card and the kernel no virtio console. Cloud Hypervisor and
+crosvm use the `0x1040`-based ids. bhyve's virtio-net and virtio-blk are
+legacy-only, an I/O BAR and no virtio capability, so a conforming driver
+matches them and refuses them by name.
+Virtio-gpu and virtio-sound have no transitional id, so their matches are
+already whole.
+
+A transitional device offers no `VIRTIO_F_ACCESS_PLATFORM` on either: QEMU
+refuses `iommu_platform=on` on a transitional device, and VirtualBox's virtio
+devices offer the bit nowhere. So behind a unit such a NIC is one no
+claim may take
+(`issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md`);
+the matches reach a card only on the path
+`issues/a-machine-without-an-iommu-refuses-every-claim.md` rules for a machine
+with no unit, and the kernel's console on every machine.
+
+**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`.
+
+**Exit**: netstack's row, its `CARDS` and the kernel's console each name both
+ids of their type; the kernel's console carries a guest boot whose console is
+QEMU's transitional `virtio-serial-pci`, and refuses the same device under
+`disable-modern=on` by name.

issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md

@@ -0,0 +1,48 @@
+---
+status: open
+kind: defect
+opened: 2026-10-12
+---
+
+# The x86 clock calibrates against an HPET, or the boot panics
+
+`arch::boot::clock` (`kernel/src/arch/x86_64/boot.rs`) takes the HPET's base
+from its ACPI table with `.expect("ACPI: HPET not found")`, and every rate the
+kernel keeps descends from that one measurement: the TSC's
+(`kernel/src/arch/x86_64/hpet.rs`) and the local APIC timer's, calibrated
+against the TSC (`apic::init_timer`). CPUID 15H/16H is read after it, as a
+cross-check only. A machine whose firmware publishes no `HPET` table stops
+there, before a timer ticks, and before PCI is enumerated, so a console that
+is a virtio function never carries the panic.
+
+QEMU's q35 has one. Of the other targets, read and not booted:
+
+| | HPET table | ACPI PM timer | TSC rate the platform states |
+|---|---|---|---|
+| Cloud Hypervisor | none | none: an `X_PM_TMR_BLK` in a FADT that sets `HW_REDUCED_ACPI`, below | CPUID `0x40000010` EAX, in kHz, where the host's TSC is invariant |
+| crosvm | none | none | CPUID 15H, where a calibrated leaf is configured |
+| VirtualBox | only with `--x86-hpet on`, default off (`VBoxManage modifyvm`) | yes | not read |
+| bhyve | yes | yes | not read |
+
+So Cloud Hypervisor, crosvm and VirtualBox without `--x86-hpet on` all end at
+this `expect`.
+
+What a machine states instead, by its own documents: the ACPI PM timer, fixed
+hardware at 3.579545 MHz (ACPI 6.5 §4.8.3.3), which q35, VirtualBox and bhyve
+publish; CPUID 15H's crystal ratio (Intel SDM Vol. 1, 253665-093US, Table 21-53),
+which the kernel already reads through `toyos_tsc::stated_hz` and which no AMD
+part states (`issues/the-loaders-lines-carry-no-time-on-an-x86-cpu-that-states-no-tsc-rate.md`);
+and a hypervisor's timing leaf `0x40000010`. crosvm without a configured leaf
+states none of them. Cloud Hypervisor states the leaf alone: its FADT writes
+`X_PM_TMR_BLK` (offset 208) but sets `HW_REDUCED_ACPI`, under which OSPM
+"will ignore fields related to the ACPI HW register interface: Fields at
+offsets 46 through 108 and 148 through 232" (ACPI 6.5 §5.2.9), the fixed
+hardware being "not implemented" there (§4.1); `toyos_acpi::fixed_hardware`
+refuses such a FADT whole (`lib/acpi/src/fadt.rs`). These ACPI sections are as
+a third-party copy of ACPI 6.6 numbers them; 6.5 itself is unread.
+
+**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`.
+
+**Exit**: a q35 boot with `-machine hpet=off` reaches `Boot: complete`, its
+`TSC:` line naming the source its rate came from; and a machine that states no
+source is refused by a line that names each one it looked for.