Functions arrive and leave: the device set, native PCIe hot-plug, Gone (lifecycle stage 2)

[Japabu]

Stage 2 of issues/a-device-function-is-driven-from-its-arrival-to-its-end.md, "Functions arrive and leave". Lands after the apertures track's stage B (#935), whose per-root declaration carries each host bridge's _OSC grant: until it is merged here, pcidev::arm_hot_plug is called with no bridge, so no slot is armed, and a_function_arrives_and_leaves reds.

The architecture was ruled by the orchestrator on this branch's design round (the ruling on points 1–7 is this PR's record): the kernel owns the slot's interrupt; arrival work runs on the device set's reader; the button means remove; one record per requester; the _OSC grant rides stage B's declaration; an unassigned BAR behind a bridge is placed by behind_bridges and proven by the register; the retired-grants exit is restated.

What changes

• The device set (toyos_abi::devset, SYS_DEVICE_SET, SYS_DEVICE_SET_READ, toyos::devset). It is a level-triggered snapshot of every PCI function the kernel hands out: a header (generation, settle time), then inventory Pci records in rising arrival order. It is minted from a SysCap carrying DEVICE, with READ and WAIT only (no DUP, no TRANSFER), so the supervisor alone holds it. devset::decode is the reader's boundary: no header, a wrong count, a non-PCI record, arrival 0, or a duplicate or out-of-order arrival is refused by name. SYS_DEVICE_INVENTORY stays inspect's.
• Records per requester (kernel/src/pcidev). The published-once table becomes up to 512 records, one per requester id, each written once and read lock-free, and each carrying its live arrival. A device that returns is a new arrival on the same record, so it gets a new number and a new handle while reusing its domain, which the last release emptied: domain ids stay bounded by requesters, not plug cycles. Every call a claim of an older arrival makes answers Gone (with_bound, the interrupt read, has_irq so a poll wakes). A departed function's release asks the device nothing: it drops the message and unmaps the grants before their pages go (tear_down_departed). It is then retired: its windows, firmware BARs, reset entry, parked grants and residue go.
• Native PCIe hot-plug (kernel/src/pcidev/slot.rs, toyos_pci::hotplug). A root or downstream port with a hot-plug-capable slot is armed only below a declared host bridge whose _OSC granted native hot-plug and capability structure control (bits 0 and 4, PCI Firmware 3.3 §4.5.1). Otherwise each port says why by name, and pcidev: N host bridge(s) declared … prints the grant per bridge.
  • The port becomes kernel-driven, its MSI-X entry armed through the claim path's remapped message, on AMD-Vi stage 6: interrupt remapping, and claims only where every unit filters SMIs #921's isolation and SMI-filter checks.
  • The handler is bounded: read and clear Slot Status, take a lost or button-removed slot's functions out of the set, at most one Slot Control write (§6.7.3.2's command completion, or its one-second allowance), post the set's watch.
  • Powering the slot, §6.6.1's 100 ms after the link is up, and reading device 0's functions below it run on the set's reader. The 100 ms reaches the supervisor as a settle time it waits to, never a spin.
  • The last release behind a removed slot powers it off, which is what QEMU's device_del waits for.
• Placement: a BAR firmware assigned no address, behind a bridge, is placed by placement::behind_bridges and accepted when the register reads the written address back with its flag bits intact. This is the one case where pcidev: a BAR behind a bridge is placed only where every bridge on its path forwards #928's "the function answers its own firmware value" proof cannot apply, and the site says so (place_unassigned). bar::kind splits a BAR's kind from its address, so an unassigned 32-bit BAR can be placed. A register that sizes to nothing is no BAR.
• IOMMU: VT-d binds a function with no context entry yet, which is any function that arrived after the unit was programmed; the displaced entry is named under domain 0. SMMUv3's unclaimable refuses a function the IORT routes were not read for. A fault from a function the fault table does not know is handed to pcidev, which clears its bus mastering and names a claim or a departed function as the owner, so it does not halt the machine.
• toyos-supervisor: a departed arrival is never started again and is forgotten once nothing runs on it. The restart budget is the address's, so a device that keeps returning is given up on like one that keeps crashing; giving up stays per arrival.
• The supervisor holds the set, watches it on its loop's poller, waits to its settle time, says PCI … joined the device set as arrival N and … (arrival N) left the device set, and reads PCI records for starts from the set (DEVICE), no longer from the inventory.
• pciholder waits on its claim until it answers something other than WouldBlock, then says what four calls answered and ends with 0.
• Issues: the track records stage 2, its gaps (physical arrival QEMU-only and why; surprise removal reached by host tests alone; ACPI-only machines unserved; no 5 s cancel window, §6.7.1.5; no bridge below a slot; SMMUv3 refuses late arrivals) and QEMU's measured slot. issues/a-released-function-never-claimed-again-keeps-its-retired-grants.md: retired grants are freed at departure; the exit for a function that stays and is never claimed again now depends on the deadline callback in issues/hotplug-blocks-a-scheduler-pass.md.

Deleted: pcidev::FUNCTIONS and arrival_of (an arrival is no longer an index), Machine::functions, VT-d bind's "every function was enumerated" assertion, and the supervisor's PCI reading from the inventory.

The guest row and why it needs QEMU

a_function_arrives_and_leaves runs on Profile::HeadlessHotplugPort: q35, an empty pcie-root-port, acpi-pci-hotplug-with-bridge-support=off. It plugs an edu over QMP, which starts a pciholder. Then device_del presses the button. The claim answers Gone on its interrupt read, config read, BAR map and DMA grant; the holder ends with 0; the supervisor says the arrival left; QEMU reports DEVICE_DELETED once the slot is off. A second plug is a new, larger arrival with a holder of its own. No bench has a hot-plug port the owner lets a driver claim behind, so physical arrival is QEMU-only; only QEMU arrives on demand. The slot's and the supervisor's decisions are host-tested.

q35's own _OSC grants what the row needs only with that flag off. The DSDT QEMU builds differs in one byte, And(CDW3, 0x1F) off versus 0x1E on, and it answers a revision other than 1 as unrecognized (evidence q35-osc/). Stage B's "q35 keeps native hot-plug" is the default machine.

Checks and evidence (ToyOSOrg/evidence pr-934, https://github.com/ToyOSOrg/evidence/tree/adcef6565dcd615b7fd2cf0d4209d0ec8439e0da/lifecycle-2)

• cargo run -- --ci host at 927635e: exit 0, 86 steps green. An earlier run at a6ab756 found two reds this branch fixed: the new config's coverage row and the port's bus-mastering site.
• cargo run -- --build-only at 927635e: x86_64 exit 0, aarch64 exit 0.
• Host tests added: toyos-supervisor (arrive, depart and return as a new arrival; a departure while a start is pending; the address's budget on a returning device), toyos_abi::devset (round trip; every malformed snapshot refused), toyos_pci::hotplug (QEMU's port read whole; the Slot Control word; what a slot is owed; what a press removes), toyos_pci::bar (an unassigned BAR placed).
• a_function_arrives_and_leaves at a6ab756 with stand-in-grant.patch, standing in for stage B's declaration: exit 0 (hotplug-row/control-green.log). Arrival 12 was held, answered Gone four times, and was retired; the second plug was arrival 13. This must be rerun on stage B's real grant after the merge.
• Negative controls, each a checked patch applied by hotplug-row/controls.sh and restored:
  • mutation-gone-unanswered reds, with no DEVICE_DELETED within 240 s.
  • mutation-arrival-reused reds, with "arrived as 12, not after 12".
• usbd_drives_the_spare, bar_map_again, bar_map_behind_a_bridge, a_drives_row_holds_one_process_per_function at a6ab756: exit 0, 4 passed (claim-rows/). Later commits are a build-gate registry, issues and The ring comparator: toyos-crypto beside ring in a guest, on metal rows (crypto stage 2) #932's merge, none in pcidev's claim path.
• QEMU's slot, read with no ToyOS involved (slot-qemu/): MSI-X only, entry 0; Slot Capabilities 0x001a007b; OVMF leaves the slot off; device_del waits for the OS to power it off.
• Metal is staged, not run: usbd_drives_the_type_c_controller (T14) and amdvi_translates_the_stick_controller (AMD). Nothing arrives on either bench. Together they establish that the boot set is unchanged and print each bridge's grant. Before stage B merges they read "0 host bridge(s) declared".

Risks and open points

• The row is red until stage B merges, and the stand-in grant is the only arming evidence until then.
• A surprise removal (presence lost) is the handler's code path with no guest reaching it: QMP presses the button.
• 512 records is a hard bound on requesters per boot; a function arriving past it is refused by name.
• Releasing a departed claim with the card still powered (button case) relies on the domain and remapping to refuse what it still issues until the slot is off; the fault is recorded against it, not the kernel.

🤖 Generated with Claude Code

https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1