From 82d46aab4ea8e7ecbfbd4147c8500db576c885b1 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 21:15:23 +0200 Subject: [PATCH 1/5] The small-kernel track says what is done, what is left, and in which order The track's text had fallen behind the tree. Every stage and step is now marked done, partly done with what is left, or not started, against the code at origin/main 97f07c381: - Done: stage 2; stage 4's steps 1, 2, 3 (the restart landed in #536), 6 and 7; stage 6 step 1; stage 7 step 1. - Partly done: stage 1 (xhci::init and gpt::probe_usb_disks still run after the supervisor's spawn and before smp::set_ready); stage 3 (met on QEMU, not on the T14's stick); stage 4's steps 4, 5, 8, 9 and 10; stage 5 (step 10's stage 1, #827); stage 6 step 2 (the instrument, #649). - Not started: stage 6 steps 3, 4 and 5; stage 7 step 2 (ps2server). Step 7 landed in #536 without the kernel USB bridge it planned: on a disk the kernel drives, fileserver opens LOG and BOOT through the kernel's partition claims. The bridge goes from the text. Step 9 splits into a file half and a block half. Its exit (BudgetExpired, DEADMAN and between_attempts gone) could not be met before step 10, because the refusal chain serves the partition claims on the stick until usbd does. The file half goes first, once libc is a file server's client; the block half lands with step 10. The programs carry their #695 names. Stage 6 step 5 loses the heartbeat clause: heartbeat.rs went in #660. The design review's account of 2026-09-25 goes, as history; the owner's rulings stay. Of the three guest tests #660 cut that the track's exits named, handler_post_without_a_pass is the kernel's and stays owed as a T14 row; home_overwrite_reads_back and apps_and_home_are_one_filesystem drove userland programs through steps, and under the owner's testing ruling (#890) do not come back. A new defect records the kernel's dependency on toyos-fat32, which no kernel source has used since #536. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...ends-on-toyos-fat32-and-uses-none-of-it.md | 21 + ...-small-interrupts-post-and-threads-wait.md | 408 ++++++++++-------- 2 files changed, 255 insertions(+), 174 deletions(-) create mode 100644 issues/the-kernel-depends-on-toyos-fat32-and-uses-none-of-it.md diff --git a/issues/the-kernel-depends-on-toyos-fat32-and-uses-none-of-it.md b/issues/the-kernel-depends-on-toyos-fat32-and-uses-none-of-it.md new file mode 100644 index 00000000000..3cccc4d3317 --- /dev/null +++ b/issues/the-kernel-depends-on-toyos-fat32-and-uses-none-of-it.md @@ -0,0 +1,21 @@ +--- +status: open +kind: defect +opened: 2026-10-12 +--- + +# The kernel depends on toyos-fat32 and uses none of it + +`kernel/Cargo.toml` lists `toyos-fat32` among the binary's dependencies +(`[target.'cfg(target_os = "none")'.dependencies]`), and no kernel source +names it: `git grep -n fat32 -- kernel` finds that manifest line alone, and +the kernel crate includes no file from outside `kernel/`. Its last user, the +kernel's FAT adapter, went in #536 and the edge stayed, so every kernel build +compiles a FAT32 driver it never calls. `lib/fat32/Cargo.toml`'s header still +says the kernel depends on it by path. + +**Exit**: `git grep -n fat32 -- kernel` finds nothing, the root `Cargo.lock` +lists no `toyos-fat32` among `kernel`'s dependencies, `lib/fat32/Cargo.toml`'s +header names no kernel, and `cargo run -- --build-only` exits 0. + +Owner: the orchestrator, which dispatches it. diff --git a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md index 8f5ef8a8139..c6b50938bca 100644 --- a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md +++ b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md @@ -10,22 +10,24 @@ Owner ruling, 2026-09-25: the kernel is to be super small, super performant and safe. This track holds that, and supersedes the ordering of `issues/every-driver-is-still-in-the-kernel.md` and `issues/a-disk-plugged-in-after-boot-is-bound-inside-a-scheduling-pass.md`, -which stay as the evidence each stage closes. +which stay as the evidence each stage closes. Owner: the orchestrator, which +dispatches each step. -The design review of 2026-09-25 read the code and found the same shape three -times: -- **Four wait mechanisms stacked** where one would do: the scheduler's - multi-waiter queue used only as a one-thread park, the completion record ring - whose records every caller discards, the userland poll registry's closed - `Source` enum with two hand-written dispatches, and ad-hoc wake paths. -- **USB with three concurrency models:** a state machine inside the scheduler - pass, disk I/O spinning with interrupts off for up to 4.75 s under one global - lock, and boot discovery calling a blocking bind from a scheduler pass. -- **Storage done busy-waiting under spinlocks:** one global VFS lock, NVMe with - one command outstanding and polled, and a 2 s operation budget "with - preemption off" that budgets audio stalls. That budget drags a refusal chain - through five layers (deadline slots, retries and backoff, `BudgetExpired`, - writeback re-enqueue, the `MID_UPDATE` task bit). +## What is left, in order + +1. **ps2server**: stage 7 step 2, with stage 6 step 3. +2. **Step 9's file half** (stage 4), once libc is a file server's client + (`issues/c-programs-name-no-file-a-file-server-holds.md`). +3. **Step 8's exit** (stage 4). Its empty view needs step 9's file half: + until then the kernel serves `/system` and `/tmp` to every process. +4. **The pcidev MSI ruling** (step 10). +5. **usbd's whole xHCI**: step 10, with stage 6 step 4, step 4's tail and + step 9's block half, which serve the stick until usbd does. +6. **Stage 6 step 5, then stage 5's windows.** + +Beside that order: stage 6 step 2's comparison, which needs only its load, +and step 5's dlopen, which waits on the same libc client as step 9's file +half. ## Stages @@ -34,131 +36,181 @@ times: its signature on the bytes in memory (the self-update's signing, so one check covers both), and hands the kernel its address and length. The kernel mounts ROOT from memory and needs no storage driver to boot. - **Exit**: the machine boots in QEMU and on the T14 with the kernel's NVMe and - USB storage paths never touched before init runs. The boot-time cost on the - stick and on NVMe is measured. - **Done** (#506; T14 run 143 green): on the stick the loader reads ROOT in - 1160 ms, the kernel reaches `Boot: complete` 1229 ms after it starts, and - the kernel issues 0 storage commands before init. The NVMe cost is not - measured yet: ToyOS is not installed on the T14's NVMe, so there is no ROOT - there to read until the self-update track installs it + **Exit**: the machine boots in QEMU and on the T14 with the kernel's NVMe + and USB storage paths never touched before the supervisor runs. The + boot-time cost on the stick and on NVMe is measured. + **Partly done** (#506; T14 run 143 green): on the stick the loader reads + ROOT in 1160 ms, the kernel reaches `Boot: complete` 1229 ms after it + starts, and the kernel issues 0 storage commands before it spawns the + supervisor. **Left**: `kernel_main` runs `xhci::init` and + `gpt::probe_usb_disks` after that spawn and before `smp::set_ready`, so + before the supervisor's first instruction + (`issues/storage-still-comes-up-before-the-supervisor-runs.md`); both go + with the kernel's xHCI at step 10. The NVMe cost waits on a ROOT on the + T14's NVMe, which the self-update track installs (`issues/the-machine-updates-itself-without-ubuntu.md`). 2. **One way to wait.** Every waitable object has one `Watch`, and a waiter is - either a thread (woken) or a user poll ring (posted). The multi-waiter queue, - the per-thread record ring, the `Source` double dispatch and the per-module - watcher lists are deleted. **Exit**: making an object waitable is one field, - and the interleaving models check the smaller protocol. - **Done** (#513): `kernel/pure/sched/watch.rs` and `park.rs`. `kernel/src` is - 720 lines smaller (62931 to 62211). No instrument in the tree measures an - interrupts-off or a preemption-off span. The nearest, `sched_check_build`'s - pass cost, separated nothing on the dev host: the largest pass per CPU was - 2.5–3.3 ms on this stage and 2.6–5.0 ms on main, three interleaved runs each. + either a thread (woken) or a user poll ring (posted). The multi-waiter + queue, the per-thread record ring, the `Source` double dispatch and the + per-module watcher lists are deleted. **Exit**: making an object waitable + is one field, and the interleaving models check the smaller protocol. + **Done** (#513): `kernel/pure/sched/watch.rs` and `park.rs`; `kernel/src` + went from 62931 to 62211 lines. 3. **Storage in userland.** NVMe and USB mass storage are programs that claim - their device through its IOMMU domain, as netd does, and serve a block + their device through its IOMMU domain, as netstack does, and serve a block protocol over shared-memory rings. Partition claims move into that service. - The kernel's NVMe driver, USB storage, block layer, page cache and the whole - budget and refusal chain are deleted. **Exit**: a crashed disk service - restarts without the kernel noticing, `/log` and `/home` survive it, and - NVMe keeps more than one command in flight. Met on QEMU at step 6 below. - **On the T14 it is met only when usbd lands** (step 10): the T14's only - writable storage is its stick, and until then that is the kernel's. + The kernel's NVMe driver, USB storage, block layer, page cache and the + whole budget and refusal chain are deleted. **Exit**: a crashed disk + service restarts without the kernel noticing, `/log` and `/home` survive + it, and NVMe keeps more than one command in flight. + **Partly done**: met on QEMU, where diskserver drives NVMe (#525, #536), + and the kernel's NVMe driver is gone (#536). **Left**: on the T14, `/log` + and `/home` are on the stick, which the kernel drives until step 10; the + kernel's USB storage, block layer, page cache and refusal chain go at + step 9. 4. **Where the data filesystem lives** — owner ruling, 2026-09-26, from the roasted proposal: userland file servers, the VFS a client library, and a - program's view a set of directory capabilities init hands it, each resolved - by its server. The page cache lives in the file server. One file server per - role (LOG, DATA, BOOT), so one crashing cannot take another down. A - file-server crash is survivable, not invisible: data survives on disk, - clients reopen, and the few states that cannot be recovered answer `Gone`; - invisible restart can come later. The kernel keeps ROOT's in-memory read - path and exec from it. USB storage stays in the kernel until stage 5 moves - the whole xHCI out, because its one IOMMU domain is shared with the - keyboard, and the panic console never depends on a userland USB program. - Ctrl+Alt+D is removed (owner, 2026-10-10: "i dont want it. i want it - removed."). No swap, declared. - `SYS_DEVICE_DMA_MAP` for zero-copy block I/O. FAT32 on `/log` in the installed product is deferred. + program's view a set of directory capabilities the supervisor hands it, + each resolved by its server. The page cache lives in the file server. One + file server per role (LOG, DATA, BOOT), so one crashing cannot take another + down. A file-server crash is survivable, not invisible: data survives on + disk, clients reopen, and the few states that cannot be recovered answer + `Gone`; invisible restart can come later. The kernel keeps ROOT's + in-memory read path and exec from it. USB storage stays in the kernel until + stage 5 moves the whole xHCI out, because its one IOMMU domain is shared + with the keyboard, and the panic console never depends on a userland USB + program. Ctrl+Alt+D is removed (owner, 2026-10-10: "i dont want it. i want + it removed."; done, #824). No swap, declared. `SYS_DEVICE_DMA_MAP` for + zero-copy block I/O. FAT32 on `/log` in the installed product is deferred. + **Partly done**: steps 1, 2, 3, 6 and 7 are done, and 4, 5, 8, 9 and 10 + partly. - Stages 3 and 4 are built as these steps, one pull request each: + Stages 3 and 4 are built as these steps: 1. **`toyos-blockring`**, the protocol, pure. **Exit**: an interleaving model of submit, complete, reset, crash and reconnect, red under a lost completion, a double completion, and a loss nothing is written again after. **Done** (#525). 2. **`SYS_DEVICE_DMA_MAP`**. **Exit**: a claimed function reaches a lent - region and nothing else, and a transfer past it or after it is taken back - is a `DMA FAULT` record. **Done** (#525). - 3. **blockd**, NVMe in userland beside the kernel's driver. **Exit**: more - than one command in flight across several queues, measured; Flush issued - when the controller has a volatile write cache; killed with a write on - the device and restarted, what was acknowledged survives, what was not is - refused, and the volume checks clean; init restarts it. **Built** (#525) - but for the last: init closes the ports of a service that ends, so the - restart is a supervisor's in the test. - 4. **Partitions in blockd**. **Exit**: a partition held by one session at a - time, the idle slot claimed by GUID and written through blockd, and - `SYS_PARTITION_READ/WRITE` deleted once no disk the kernel drives serves - a claim. **Built** (#525) but for the last: the kernel's disks, the stick - among them, still serve claims until step 10, and a `part:` row still - mints a kernel claim. - 5. **Spawn and dlopen from an image handle.** **Exit**: a program in `/apps` - runs having been read by userland, its faults served from the image - object and not the VFS. - 6. **fsd for DATA over blockd**, with the `toyos::fs` client the std fork - uses; the kernel's DATA mount deleted with it. **Exit**: + region and nothing else, and a transfer past it or after it is taken + back is a `DMA FAULT` record. **Done** (#525). + 3. **diskserver**, NVMe in userland. **Exit**: more than one command in + flight across several queues, measured; Flush issued when the + controller has a volatile write cache; killed with a write on the + device and restarted, what was acknowledged survives, what was not is + refused, and the volume checks clean; the supervisor restarts it. + **Done** (#525; the restart, #536). + 4. **Partitions in diskserver**. **Exit**: a partition held by one session + at a time, the idle slot claimed by GUID and written through + diskserver, and `SYS_PARTITION_READ/WRITE` deleted once no disk the + kernel drives serves a claim. **Partly done** (#525, #826): sessions, a + block connector that opens only the partitions its badge grants, and + the idle slot on diskserver's disk written through one. **Left**: on a + disk the kernel drives, the stick among them, the supervisor still + claims a role's and the idle slot's partitions from the kernel, and + `SYS_PARTITION_READ/WRITE` serve them; both go with the kernel's USB + storage at step 10. + 5. **Spawn and dlopen from an image handle.** **Exit**: a program in + `/apps` runs having been read by userland, its faults served from the + image object and not the VFS. **Partly done** (#536): spawn + (`SpawnArgs::image`). **Left**: dlopen. `SYS_DLOPEN` opens a path the + kernel serves, so no library on a file server loads + (`issues/a-package-cannot-ship-its-own-libraries.md`); #536 left + dlopen from an image out of the ABI, since nothing calls it before libc + is a file server's client. Its exit: a library in `/apps` loads having been read + by userland, its faults served from the image object. + 6. **fileserver for DATA over diskserver**, with the `toyos::fs` client the + std fork uses; the kernel's DATA mount deleted with it. **Exit**: `home_overwrite_reads_back` and `apps_and_home_are_one_filesystem` - green; fsd killed mid-write, restarted, the volume mounts, fsync'd data - reads back on the host, and the kernel log does not change. - 7. **fsd for LOG and BOOT; logd on the client library.** On the T14 the - stick is still the kernel's, so this step builds **the kernel USB - bridge**: the kernel's USB mass storage serves `toyos-blockring` - sessions, one per partition held, as blockd does, and fsd opens LOG - through it. It is the one block server the kernel keeps, a compromise - with an exit of its own: deleted at step 10, when usbd serves the same - sessions. **Exit**: the log's durability tests without `WouldBlock`, on - QEMU over blockd and on the T14 over the bridge; the quiesce cases green - with the storage chain kept running; and the panic wait still publishes - the last line. - 8. **Views as capabilities**, the isolation track's stage 1. **Exit**: that - stage's escape suite, run against fsd. - 9. **Delete the kernel storage stack**: the VFS down to ROOT's resolver, - both writable adapters, both caches, write-back, durability, tmpfs, the - block layer, GPT, the NVMe driver, the refusal chain and the file - syscalls. **Exit**: `BudgetExpired`, `DEADMAN` and - `between_attempts` appear nowhere, and the kernel's lines and longest - interrupts-off and preemption-off windows are measured. - 10. **usbd**, stage 5's second half: the whole xHCI moves, HID to the - keyboard claim and mass storage over `toyos-blockring`, and the kernel - USB bridge is deleted. A USB keyboard's transitions reach the kernel - only as usbd's keyboard claim. **Exit**, on the T14: `/log` survives - usbd killed mid-batch, and the keyboard keeps working while a stick - misbehaves. QEMU's xHCI keeps its MSI-X table in the - BAR that holds the registers, which refuses usbd's claim as it refuses - blockd's - (`issues/a-controller-whose-msix-table-is-in-bar-0-cannot-be-driven-from-userland.md`), - and the T14's has none on its capability walk, whose end is not read - (`issues/every-driver-is-still-in-the-kernel.md`). Its decoders are - `toyos-xhci`'s Bulk-Only and SCSI modules and `toyos-usbhid`. + green; fileserver killed mid-write, restarted, the volume mounts, + fsync'd data reads back on the host, and the kernel log does not + change. **Done** (#536). #660 cut both guest tests, and under the + owner's testing ruling (#890: "we try not to test integration between + modules unless its the kernel") neither comes back: each started + userland programs, drove them through writes and a shutdown, and read + the disk on the host, so what they checked is fileserver's and + `bcachefs`'s to hold in module tests. The guest-suite track's stage D + still lists both. + 7. **fileserver for LOG and BOOT; logkeeper on the client library.** + **Exit**: the log's durability tests without `WouldBlock`, the quiesce + cases green with the storage chain kept running, and the panic wait + still publishes the last line. **Done** (#536), without the kernel USB + bridge this step had planned: on a disk the kernel drives, fileserver + opens LOG and BOOT through the kernel's partition claims, which step 10 + deletes with the kernel's USB storage. + 8. **Views as capabilities**, the isolation track's stage 1 + (`issues/every-program-sees-only-the-files-it-was-given.md`). + **Exit**: that stage's, run against fileserver: an escape suite red on + a resolver that walks the global tree and green on fileserver's, and a + process given an empty view naming nothing. **Partly done** (#729, + #742): a connection is the grant the supervisor minted on its role's + one port, and `fs_escape` is refused a symlink that climbs out, a `..` + on the wire and a directory it was not given. **Left**: the rest of the + escape suite, a rename racing a lookup among it, shown red on a + global-tree resolver; and the empty view, which names `/system` and + `/tmp` while the kernel serves them (step 9's file half). + 9. **Delete the kernel storage stack.** **Partly done** (#536): the NVMe + driver, both writable adapters, write-back, durability and the + `MID_UPDATE` bit are gone. Two halves are left, and they land apart: + - **The file half**: the VFS down to ROOT's resolver, tmpfs (`tmpfs.rs`, + `fs_rename.rs`), the page cache (`file_cache.rs`) and the file + syscalls. `/tmp` leaves the kernel per session, as the owner ruled + ("Per session", 2026-10-03, in the isolation track), and `/system` is + read over the file protocol, which + `issues/c-programs-name-no-file-a-file-server-holds.md`'s exit puts + after libc is a file server's client. **Exit**: `tmpfs.rs`, + `fs_rename.rs` and `file_cache.rs` are gone, `vfs.rs` resolves ROOT + for exec alone, and `SYS_OPEN`, `SYS_READDIR`, `SYS_RENAME`, + `SYS_MKDIR` and the rest of the path family are gone from `sdk/abi`. + - **The block half**, with step 10: the block layer (`block.rs`), GPT + (`gpt.rs`), USB storage (`drivers/usb_storage.rs`), the refusal chain + (`BudgetExpired`, `until_answered`, `DEADMAN`, `between_attempts`) and + `SYS_PARTITION_READ/WRITE`, all of which serve the stick until usbd + does. **Exit**: `BudgetExpired`, `DEADMAN` and `between_attempts` + appear nowhere in the tree. - **Stage 1, usbd beside the kernel's driver**: the test-only + Each half measures the kernel's lines and its longest interrupts-off + and preemption-off windows, by stage 6 step 2's rule. + 10. **usbd**, stage 5's second half: the whole xHCI moves, HID to the + keyboard claim and mass storage over `toyos-blockring`. A USB + keyboard's transitions reach the kernel only as usbd's keyboard claim. + Its decoders are `toyos-xhci`'s Bulk-Only and SCSI modules and + `toyos-usbhid`. **Exit**, on the T14: `/log` survives usbd killed + mid-batch, and the keyboard keeps working while a stick misbehaves. + **Partly done** (#827), its stage 1: the test-only `xhci-leave=:` leaves one controller to a claim, and `/system/bin/usbd` drives it — the handoff, a reset, its rings in one grant, MSI through the claim, and every device enumerated and named - over `inspect`; no class is bound. QEMU's spare is - `qemu-xhci,msix=off,msi=on`: with MSI-X off and MSI left `auto` it - offers no MSI either, and the claim is refused for it. Read on the T14 - (stage 0, `boot:testcases` readbacks): firmware owns neither - controller. 00:0d.0, 8086:9a13, the Type-C one (port 1 USB 2.0, ports - 2..=5 USB 3.1) and 00:14.0, 8086:a0ed (ports 1..=12 USB 2.0, 13..=16 - USB 3.1) each read `USBLEGSUP 0x01002201` and `USBLEGCTLSTS - 0xe0000000` — firmware's semaphore clear, every SMI enable clear, the - three status bits latched — and each is armed on MSI. Neither - 00:0d.0, whose release `usbd_drives_the_type_c_controller` reads as - `reset by nothing (Express: no capability; AF: no capability; PM: - No_Soft_Reset set)`, nor QEMU's `qemu-xhci` advertises a reset, so a - claim on either inherits the last holder's ranges as residue; which - 00:14.0 advertises no boot has printed. The orchestrator's stage-0 rulings for the - stages after: pcidev arms MSI and keeps MSI-X Enable clear where the - table shares the register BAR, and one usbd runs per controller. - Two compromises until this step's whole-xHCI move, owned by it, whose - exit is the kernel's copy deleted with the kernel's driver. + over `inspect`; no class is bound, and no shipped image runs it. + **Left**, first the orchestrator's stage-0 ruling that pcidev arms MSI + and keeps MSI-X Enable clear where the table shares the register BAR, + then the whole xHCI, one usbd per controller (the orchestrator's other + stage-0 ruling). pcidev today arms MSI only on a function with no MSI-X, and + maps no BAR that holds the table (`kernel/src/pcidev/mod.rs`). QEMU's + xHCI keeps its MSI-X table in the BAR that holds the registers, which + refuses usbd's claim as it refuses diskserver's + (`issues/a-controller-whose-msix-table-is-in-bar-0-cannot-be-driven-from-userland.md`), + and the T14's has none on its capability walk, whose end is not read + (`issues/every-driver-is-still-in-the-kernel.md`). The ruling's exit: + usbd's claim on QEMU's `qemu-xhci` with its MSI-X left on is handed + over armed on MSI, with MSI-X Enable clear. + + QEMU's spare today is `qemu-xhci,msix=off,msi=on`: with MSI-X off and + MSI left `auto` it offers no MSI either, and the claim is refused for + it. Read on the T14 (stage 0, `boot:testcases` readbacks): firmware + owns neither controller. 00:0d.0, 8086:9a13, the Type-C one (port 1 + USB 2.0, ports 2..=5 USB 3.1) and 00:14.0, 8086:a0ed (ports 1..=12 USB + 2.0, 13..=16 USB 3.1) each read `USBLEGSUP 0x01002201` and + `USBLEGCTLSTS 0xe0000000` — firmware's semaphore clear, every SMI + enable clear, the three status bits latched — and each is armed on + MSI. Neither 00:0d.0, whose release + `usbd_drives_the_type_c_controller` reads as `reset by nothing + (Express: no capability; AF: no capability; PM: No_Soft_Reset set)`, + nor QEMU's `qemu-xhci` advertises a reset, so a claim on either + inherits the last holder's ranges as residue; which reset 00:14.0 + advertises no boot has printed. + + Two compromises until this step, owned by it, whose exit is the + kernel's copy deleted with the kernel's driver. `toyos-xhci`'s `descriptor` and the kernel's `parse_config` (`kernel/src/drivers/xhci/device.rs`) are two decoders of the same device-chosen bytes with different refusal rules — the kernel ends its @@ -170,68 +222,74 @@ times: orchestrator ruled the kernel's copy is not moved onto the crate's, since it goes whole here, and its `xhci-xecp-selftest` actuator goes with it. -5. **USB by userland**, with discovery and recovery - written once as straight-line code. **Exit**: no interrupts-off window - longer than a register access, and keyboard input keeps flowing while a - stick misbehaves. -6. A handler posts its - device's `Watch` and ends its interrupt; the thread waiting on that watch - does the work, and no step creates a kernel thread. `irq_ring`, the driver - list in `drain_irqs` and the idle loop's device checks are gone by step 5. - Each step measures the kernel's lines against stage 6's first commit, and - the longest interrupts-off and preemption-off windows against the readings - step 2's exit puts in the tree, by that exit's rule. - Steps 3 and 4 do not land alone: they land with stage 7.2 and - with usbd. +5. **USB by userland**, with discovery and recovery written once as + straight-line code: step 10, and this exit. **Exit**: no interrupts-off + window longer than a register access, read by stage 6 step 2's + instrument, and keyboard input keeps flowing while a stick misbehaves. + **Partly done**: step 10's stage 1 (#827). **Left**: the rest of step 10, + then this exit. +6. A handler posts its device's `Watch` and ends its interrupt; the thread + waiting on that watch does the work, and no step creates a kernel thread. + `irq_ring`, the driver list in `drain_irqs` and the idle loop's device + checks are gone by step 5. Each step measures the kernel's lines against + stage 6's first commit, and the longest interrupts-off and preemption-off + windows against the readings step 2's exit puts in the tree, by that + exit's rule. Steps 3 and 4 do not land alone: they land with stage 7 step 2 + and with usbd. **Partly done**: step 1, and step 2's instrument. 1. **Interrupts post.** A post is legal in a handler: the watches a handler posts, and the completions of a ring they complete into, sit behind interrupts-off locks nothing allocates or frees under, and every other watch's lock leaves interrupts open. A claimed function's vector, the IOMMU's refusal and both audio backends post from the handler, and `irq_ring`'s `UserDev` and `Audio` and their arms in `drain_irqs` go. - The thread is the holder's: netd's, blockd's, soundd's mix thread, and - an `isa` claim's holder. **Exit**: `handler_post_without_a_pass`, - a vector taken on a CPU holding preemption off, inside a post of its own - watch, inside a completion into a ring polling it, or inside that ring's - own watch, posting once that section lets go and before any pass, red on - the base; the watch's loom models over the new post. - 2. **The windows, measured**: the longest interrupts-off and preemption-off - windows per CPU, reported beside the IRQ census and fed by each - architecture's masking primitives and entries, the number the ARM - track's stage 4 owes as well. Built: the `mask-windows` kernel and the - T14's `mask_windows` row. The exit's first half is met: the instrument - reads back a window of known length on the T14, which is x86 metal, and - the row's judge refuses a boot that reads it back shorter than it was - held or at more than twice that. **Exit**, open: neither window is - longer with step 1 than - with it reverted on the tree that carries the instrument, the patch - posted on #649 being what reverted means. That comparison is the median - of at least five interleaved boots an arm, each read from the load's own - report with a report taken as the load starts, against a tolerance - stated before measuring, the reverted arm's own spread, with the tail - reported beside it, under a load in which a handler step 1 changed posts - into a watch that reaches a ring with parked threads + The thread is the holder's: netstack's, diskserver's, soundserver's mix + thread, and an `isa` claim's holder. **Exit**: + `handler_post_without_a_pass`, a vector taken on a CPU holding + preemption off, inside a post of its own watch, inside a completion + into a ring polling it, or inside that ring's own watch, posting once + that section lets go and before any pass, red on the base; the watch's + loom models over the new post. **Done** (#634). #660 cut + `handler_post_without_a_pass`; the guest-suite track's stage C owes it + as a T14 row, and the owner's testing ruling (#890) keeps it, since what + it checks is the kernel's alone. + 2. **The windows, measured**: the longest interrupts-off and + preemption-off windows per CPU, reported beside the IRQ census and fed + by each architecture's masking primitives and entries, the number the + ARM track's stage 4 owes as well. **Partly done** (#649): the + `mask-windows` kernel and the T14's `mask_windows` row; the instrument + reads back a window of known length on the T14, which is x86 metal, + and the row's judge refuses a boot that reads it back shorter than it + was held or at more than twice that. **Left**, the exit: neither window + is longer with step 1 than with it reverted on the tree that carries + the instrument, the patch posted on #649 being what reverted means. + That comparison is the median of at least five interleaved boots an + arm, each read from the load's own report with a report taken as the + load starts, against a tolerance stated before measuring, the reverted + arm's own spread, with the tail reported beside it, under a load in + which a handler step 1 changed posts into a watch that reaches a ring + with parked threads (`issues/a-process-lengthens-an-interrupts-off-walk-by-the-threads-it-parks-on-one-ring.md`'s second bullet). `ring_park_herd` is not that load: its walks start in `SYS_INBOX_SUBMIT`, which runs with interrupts masked on both arms, and it reaches no handler step 1 changed. - 3. **The i8042's thread is ps2server's** (stage 7.2): `irq_ring`'s + 3. **The i8042's thread is ps2server's** (stage 7 step 2): `irq_ring`'s `I8042`, `keyboard_controller::service` and the idle loop's - `verdict_due` go with the kernel's driver. **Exit**: that stage's. - 4. **xHCI's thread is usbd's** (step 10 above): `Xhci`, `poll_if_pending` - and `port_work_pending` go with the kernel's driver, and `irq_ring` and - `sync::OwedLock`, whose one user is `XHCI`, with them. **Exit**: step 10's. - 5. **The pass is the scheduler's.** `drain_irqs` goes: the heartbeat - becomes `pass`'s own, and the TCO feed stays, since what it proves is - that passes run. **Exit**: `drain_irqs` and the - idle loop's device checks are gone, both windows are measured against - step 2's readings by its rule, and the exits of - `issues/an-irq-watchs-freeing-cancel-compiles-in-a-handler.md` - and + `verdict_due` go with the kernel's driver. **Exit**: that step's. + **Not started.** + 4. **xHCI's thread is usbd's** (stage 4 step 10): `Xhci`, + `poll_if_pending` and `port_work_pending` go with the kernel's driver, + and `irq_ring` and `sync::OwedLock`, whose one user is `XHCI`, with + them. **Exit**: step 10's. **Not started.** + 5. **The pass is the scheduler's.** `drain_irqs` goes, and the TCO feed + stays on the pass, since what it proves is that passes run. **Exit**: + `drain_irqs` and the idle loop's device checks are gone, both windows + are measured against step 2's readings by its rule, and the exits of + `issues/an-irq-watchs-freeing-cancel-compiles-in-a-handler.md` and `issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md` - are met. + are met. **Not started.** 7. **The i8042 leaves the kernel.** Owner ruling, 2026-09-28: drivers are userland, and a dead kernel takes no input, with no emergency way. + **Partly done**: step 1. 1. **The panic console takes no input, and an `isa` claim grants a process exact ports through the TSS I/O permission bitmap and its ISA lines as records** (`kernel/src/isa.rs`). **Done** (#592). @@ -239,9 +297,11 @@ times: and mouse streams so the merge with USB HID stays where it is; the kernel's driver, its vector, its actuators and the `keyboard_controller` seam deleted. A keyboard claim is refused while no - source exists, which init's order of endowment then decides. **Exit**: - no i8042 code in the kernel, and typing resumes after ps2server is killed - and restarted. + source exists, which the supervisor's order of endowment then decides. + **Exit**: no i8042 code in `kernel/src` (`arch/x86_64/i8042/`, + `arch/x86_64/idt/i8042.rs` and both architectures' + `keyboard_controller` gone), and typing resumes after ps2server is + killed and restarted. **Not started.** ## Standing From a29c4735a476886e2ebe9b6a523345857a410059 Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 01:21:16 +0200 Subject: [PATCH 2/5] The small-kernel track answers review round 1 of #900 Exits hold the kernel's part of a restart and name the module tests that hold each program's (#890); tests owed are recorded with an owner and an exit. P1 of the Wi-Fi track replaces the stage-0 MSI ruling for usbd's claim, as the orchestrator ruled on #900. Done steps shrink to their titles and pull requests. The path family's numbers are listed, the block half's exit is scoped to kernel/, 7.2 names the toyos-ps2 edge, and the fat32 issue names the two build comments that make its false claim. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...ends-on-toyos-fat32-and-uses-none-of-it.md | 10 +- ...-small-interrupts-post-and-threads-wait.md | 238 +++++++++--------- 2 files changed, 125 insertions(+), 123 deletions(-) diff --git a/issues/the-kernel-depends-on-toyos-fat32-and-uses-none-of-it.md b/issues/the-kernel-depends-on-toyos-fat32-and-uses-none-of-it.md index 3cccc4d3317..8f236dab041 100644 --- a/issues/the-kernel-depends-on-toyos-fat32-and-uses-none-of-it.md +++ b/issues/the-kernel-depends-on-toyos-fat32-and-uses-none-of-it.md @@ -11,11 +11,13 @@ opened: 2026-10-12 names it: `git grep -n fat32 -- kernel` finds that manifest line alone, and the kernel crate includes no file from outside `kernel/`. Its last user, the kernel's FAT adapter, went in #536 and the edge stayed, so every kernel build -compiles a FAT32 driver it never calls. `lib/fat32/Cargo.toml`'s header still -says the kernel depends on it by path. +compiles a FAT32 driver it never calls. Three comments still say the kernel +uses it: `lib/fat32/Cargo.toml`'s header (the kernel depends on it by path), +`build/fatformat.rs`'s module doc ("the kernel's driver") and `populate`'s +doc in `build/image.rs` ("the driver the kernel appends `kernel.log` with"). **Exit**: `git grep -n fat32 -- kernel` finds nothing, the root `Cargo.lock` -lists no `toyos-fat32` among `kernel`'s dependencies, `lib/fat32/Cargo.toml`'s -header names no kernel, and `cargo run -- --build-only` exits 0. +lists no `toyos-fat32` among `kernel`'s dependencies, none of the three +comments names the kernel, and `cargo run -- --build-only` exits 0. Owner: the orchestrator, which dispatches it. diff --git a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md index c6b50938bca..b9028e9f75e 100644 --- a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md +++ b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md @@ -11,7 +11,9 @@ and safe. This track holds that, and supersedes the ordering of `issues/every-driver-is-still-in-the-kernel.md` and `issues/a-disk-plugged-in-after-boot-is-bound-inside-a-scheduling-pass.md`, which stay as the evidence each stage closes. Owner: the orchestrator, which -dispatches each step. +dispatches each step and each test this track owes. An exit holds the +kernel's part of a restart, and named module tests hold each program's +(#890). ## What is left, in order @@ -20,14 +22,20 @@ dispatches each step. (`issues/c-programs-name-no-file-a-file-server-holds.md`). 3. **Step 8's exit** (stage 4). Its empty view needs step 9's file half: until then the kernel serves `/system` and `/tmp` to every process. -4. **The pcidev MSI ruling** (step 10). +4. **P1 of the Wi-Fi track** (`issues/there-is-no-wifi.md`): a BAR stays + where firmware put it, handed over in 4 KiB pages with its MSI-X table's + page, under the owner's Q2 ruling. The orchestrator ruled on #900 that + this route serves usbd's claim (step 10). 5. **usbd's whole xHCI**: step 10, with stage 6 step 4, step 4's tail and step 9's block half, which serve the stick until usbd does. 6. **Stage 6 step 5, then stage 5's windows.** -Beside that order: stage 6 step 2's comparison, which needs only its load, -and step 5's dlopen, which waits on the same libc client as step 9's file -half. +Beside that order, each waiting on something outside it: +- stage 1's NVMe boot-time measurement, on a ROOT on the T14's NVMe, which + the self-update track installs + (`issues/the-machine-updates-itself-without-ubuntu.md`); +- stage 6 step 2's comparison, on its load; +- step 5's dlopen, on the same libc client as step 9's file half. ## Stages @@ -39,35 +47,33 @@ half. **Exit**: the machine boots in QEMU and on the T14 with the kernel's NVMe and USB storage paths never touched before the supervisor runs. The boot-time cost on the stick and on NVMe is measured. - **Partly done** (#506; T14 run 143 green): on the stick the loader reads - ROOT in 1160 ms, the kernel reaches `Boot: complete` 1229 ms after it - starts, and the kernel issues 0 storage commands before it spawns the - supervisor. **Left**: `kernel_main` runs `xhci::init` and + **Partly done** (#506; T14 run 143 green), the stick's cost measured. + **Left**: `kernel_main` runs `xhci::init` and `gpt::probe_usb_disks` after that spawn and before `smp::set_ready`, so before the supervisor's first instruction (`issues/storage-still-comes-up-before-the-supervisor-runs.md`); both go - with the kernel's xHCI at step 10. The NVMe cost waits on a ROOT on the - T14's NVMe, which the self-update track installs - (`issues/the-machine-updates-itself-without-ubuntu.md`). -2. **One way to wait.** Every waitable object has one `Watch`, and a waiter is - either a thread (woken) or a user poll ring (posted). The multi-waiter - queue, the per-thread record ring, the `Source` double dispatch and the - per-module watcher lists are deleted. **Exit**: making an object waitable - is one field, and the interleaving models check the smaller protocol. - **Done** (#513): `kernel/pure/sched/watch.rs` and `park.rs`; `kernel/src` - went from 62931 to 62211 lines. + with the kernel's xHCI at step 10. The NVMe cost waits on the self-update + track (above). +2. **One way to wait.** **Done** (#513). 3. **Storage in userland.** NVMe and USB mass storage are programs that claim their device through its IOMMU domain, as netstack does, and serve a block protocol over shared-memory rings. Partition claims move into that service. The kernel's NVMe driver, USB storage, block layer, page cache and the - whole budget and refusal chain are deleted. **Exit**: a crashed disk - service restarts without the kernel noticing, `/log` and `/home` survive - it, and NVMe keeps more than one command in flight. - **Partly done**: met on QEMU, where diskserver drives NVMe (#525, #536), - and the kernel's NVMe driver is gone (#536). **Left**: on the T14, `/log` - and `/home` are on the stick, which the kernel drives until step 10; the - kernel's USB storage, block layer, page cache and refusal chain go at - step 9. + whole budget and refusal chain are deleted. **Exit**: NVMe keeps more + than one command in flight, measured; a dead holder's claim is handed + over again, held by `usbd_drives_the_spare` (guest) and + `usbd_drives_the_type_c_controller` (T14); and what a disk service's + death leaves is held by `toyos-blockring`'s + `every_request_is_answered_exactly_once`, + `what_a_flush_calls_durable_is_on_the_medium` and + `what_was_on_the_wire_at_the_end_is_refused_and_what_was_not_waits`, by + fileserver's + `a_directory_rename_is_whole_under_one_name_wherever_the_server_is_killed` + for DATA, and for LOG by the FAT test step 10 owes. **Partly done** + (#525, #536): diskserver drives NVMe, and the kernel's NVMe driver is + gone. **Left**: on the T14, `/log` and `/home` are on the + stick, which the kernel drives until step 10; the kernel's USB storage, + block layer, page cache and refusal chain go at step 9. 4. **Where the data filesystem lives** — owner ruling, 2026-09-26, from the roasted proposal: userland file servers, the VFS a client library, and a program's view a set of directory capabilities the supervisor hands it, @@ -86,19 +92,9 @@ half. partly. Stages 3 and 4 are built as these steps: - 1. **`toyos-blockring`**, the protocol, pure. **Exit**: an interleaving - model of submit, complete, reset, crash and reconnect, red under a lost - completion, a double completion, and a loss nothing is written again - after. **Done** (#525). - 2. **`SYS_DEVICE_DMA_MAP`**. **Exit**: a claimed function reaches a lent - region and nothing else, and a transfer past it or after it is taken - back is a `DMA FAULT` record. **Done** (#525). - 3. **diskserver**, NVMe in userland. **Exit**: more than one command in - flight across several queues, measured; Flush issued when the - controller has a volatile write cache; killed with a write on the - device and restarted, what was acknowledged survives, what was not is - refused, and the volume checks clean; the supervisor restarts it. - **Done** (#525; the restart, #536). + 1. **`toyos-blockring`**, the protocol, pure. **Done** (#525). + 2. **`SYS_DEVICE_DMA_MAP`**. **Done** (#525). + 3. **diskserver**, NVMe in userland. **Done** (#525; the restart, #536). 4. **Partitions in diskserver**. **Exit**: a partition held by one session at a time, the idle slot claimed by GUID and written through diskserver, and `SYS_PARTITION_READ/WRITE` deleted once no disk the @@ -114,29 +110,26 @@ half. image object and not the VFS. **Partly done** (#536): spawn (`SpawnArgs::image`). **Left**: dlopen. `SYS_DLOPEN` opens a path the kernel serves, so no library on a file server loads - (`issues/a-package-cannot-ship-its-own-libraries.md`); #536 left - dlopen from an image out of the ABI, since nothing calls it before libc - is a file server's client. Its exit: a library in `/apps` loads having been read - by userland, its faults served from the image object. - 6. **fileserver for DATA over diskserver**, with the `toyos::fs` client the - std fork uses; the kernel's DATA mount deleted with it. **Exit**: - `home_overwrite_reads_back` and `apps_and_home_are_one_filesystem` - green; fileserver killed mid-write, restarted, the volume mounts, - fsync'd data reads back on the host, and the kernel log does not - change. **Done** (#536). #660 cut both guest tests, and under the - owner's testing ruling (#890: "we try not to test integration between - modules unless its the kernel") neither comes back: each started - userland programs, drove them through writes and a shutdown, and read - the disk on the host, so what they checked is fileserver's and - `bcachefs`'s to hold in module tests. The guest-suite track's stage D - still lists both. + (`issues/a-package-cannot-ship-its-own-libraries.md`). Its exit: a + library in `/apps` loads having been read by userland, its faults + served from the image object, and `SYS_DLOPEN` takes no path. + 6. **fileserver for DATA over diskserver.** **Done** (#536). #660 cut + its guest tests and #890 keeps them cut. + `apps_and_home_are_one_filesystem`'s claim, `/apps` and `/home` one + volume, is held by fileserver's + `a_directory_rename_is_whole_under_one_name_wherever_the_server_is_killed`, + which renames `home/staged` to `apps/pkg` in one volume. + `home_overwrite_reads_back`'s, a recorded failure (`e2bd52e83`: a + same-length `/home` overwrite read back 0 bytes), nothing holds. + **Owed**, by the orchestrator, waiting on nothing: a test in + `system/fileserver/src/data.rs` that holds a synced file open, + reopens it with truncate, writes as many other bytes, closes the held + handle then the writer, syncs, remounts and reads the new bytes back + whole, shown red on an `open` whose truncate makes a new file under a + held file's name. 7. **fileserver for LOG and BOOT; logkeeper on the client library.** - **Exit**: the log's durability tests without `WouldBlock`, the quiesce - cases green with the storage chain kept running, and the panic wait - still publishes the last line. **Done** (#536), without the kernel USB - bridge this step had planned: on a disk the kernel drives, fileserver - opens LOG and BOOT through the kernel's partition claims, which step 10 - deletes with the kernel's USB storage. + **Done** (#536); on a disk the kernel drives, fileserver opens LOG and + BOOT through the kernel's partition claims, which step 10 deletes. 8. **Views as capabilities**, the isolation track's stage 1 (`issues/every-program-sees-only-the-files-it-was-given.md`). **Exit**: that stage's, run against fileserver: an escape suite red on @@ -148,9 +141,8 @@ half. escape suite, a rename racing a lookup among it, shown red on a global-tree resolver; and the empty view, which names `/system` and `/tmp` while the kernel serves them (step 9's file half). - 9. **Delete the kernel storage stack.** **Partly done** (#536): the NVMe - driver, both writable adapters, write-back, durability and the - `MID_UPDATE` bit are gone. Two halves are left, and they land apart: + 9. **Delete the kernel storage stack.** **Partly done** (#536). Two + halves are left, and they land apart: - **The file half**: the VFS down to ROOT's resolver, tmpfs (`tmpfs.rs`, `fs_rename.rs`), the page cache (`file_cache.rs`) and the file syscalls. `/tmp` leaves the kernel per session, as the owner ruled @@ -159,14 +151,16 @@ half. `issues/c-programs-name-no-file-a-file-server-holds.md`'s exit puts after libc is a file server's client. **Exit**: `tmpfs.rs`, `fs_rename.rs` and `file_cache.rs` are gone, `vfs.rs` resolves ROOT - for exec alone, and `SYS_OPEN`, `SYS_READDIR`, `SYS_RENAME`, - `SYS_MKDIR` and the rest of the path family are gone from `sdk/abi`. + for exec and `SYS_DLOPEN` alone, and `sdk/abi` has no `SYS_OPEN` (9), + `SYS_STAT` (16), `SYS_READDIR` (17), `SYS_DELETE` (18), `SYS_CHDIR` + (20), `SYS_GETCWD` (21), `SYS_RENAME` (52), `SYS_MKDIR` (53), + `SYS_RMDIR` (54), `SYS_SYMLINK` (81) or `SYS_READLINK` (82). + `SYS_DLOPEN` (55) is step 5's. - **The block half**, with step 10: the block layer (`block.rs`), GPT (`gpt.rs`), USB storage (`drivers/usb_storage.rs`), the refusal chain - (`BudgetExpired`, `until_answered`, `DEADMAN`, `between_attempts`) and - `SYS_PARTITION_READ/WRITE`, all of which serve the stick until usbd - does. **Exit**: `BudgetExpired`, `DEADMAN` and `between_attempts` - appear nowhere in the tree. + and `SYS_PARTITION_READ/WRITE`, all of which serve the stick until + usbd does. **Exit**: `git grep -n -e BudgetExpired -e DEADMAN -e + between_attempts -e until_answered -- kernel` finds nothing. Each half measures the kernel's lines and its longest interrupts-off and preemption-off windows, by stage 6 step 2's rule. @@ -174,31 +168,45 @@ half. keyboard claim and mass storage over `toyos-blockring`. A USB keyboard's transitions reach the kernel only as usbd's keyboard claim. Its decoders are `toyos-xhci`'s Bulk-Only and SCSI modules and - `toyos-usbhid`. **Exit**, on the T14: `/log` survives usbd killed - mid-batch, and the keyboard keeps working while a stick misbehaves. + `toyos-usbhid`. **Exit**: the kernel's part, usbd's claims handed + over again at its death: the controller's by `usbd_drives_the_spare` + on the default `qemu-xhci` (below) and + `usbd_drives_the_type_c_controller`, the keyboard's by stage 7 step + 2's kernel test on usbd's claim. usbd's, a device that stops answering + costing only itself: `toyos-xhci-sim`'s + `a_controller_that_never_answers_still_releases_the_port`, + `a_recovery_the_controller_refuses_lets_the_device_go` and + `no_sequence_breaks_an_invariant`, with one sim test this step owes: + a stick whose bulk endpoint never answers leaves a keyboard on the + same controller delivering every report, red on a usbd that waits on + the stick before it reads the keyboard's ring. `/log`'s, what a killed + usbd had on the wire: stage 3's `toyos-blockring` tests, with one test + this step owes in `system/fileserver/src/fat.rs`: a disk stopped after + each block write in turn of an append and its sync mounts at every + stop and reads back all the last whole sync held. + **Partly done** (#827), its stage 1: the test-only `xhci-leave=:` leaves one controller to a claim, and `/system/bin/usbd` drives it — the handoff, a reset, its rings in one grant, MSI through the claim, and every device enumerated and named over `inspect`; no class is bound, and no shipped image runs it. - **Left**, first the orchestrator's stage-0 ruling that pcidev arms MSI - and keeps MSI-X Enable clear where the table shares the register BAR, - then the whole xHCI, one usbd per controller (the orchestrator's other - stage-0 ruling). pcidev today arms MSI only on a function with no MSI-X, and - maps no BAR that holds the table (`kernel/src/pcidev/mod.rs`). QEMU's - xHCI keeps its MSI-X table in the BAR that holds the registers, which - refuses usbd's claim as it refuses diskserver's - (`issues/a-controller-whose-msix-table-is-in-bar-0-cannot-be-driven-from-userland.md`), - and the T14's has none on its capability walk, whose end is not read - (`issues/every-driver-is-still-in-the-kernel.md`). The ruling's exit: - usbd's claim on QEMU's `qemu-xhci` with its MSI-X left on is handed - over armed on MSI, with MSI-X Enable clear. + **Left**, first P1 (order item 4), then the whole xHCI, one usbd per + controller (the orchestrator's stage-0 ruling). pcidev today arms MSI + only on a function with no MSI-X, and maps no BAR that holds the + table (`kernel/src/pcidev/mod.rs`). QEMU's xHCI keeps its MSI-X table + in the BAR that holds the registers, which refuses usbd's claim as it + refuses diskserver's + (`issues/a-controller-whose-msix-table-is-in-bar-0-cannot-be-driven-from-userland.md`, + whose exit P1 rewrites), so QEMU's spare today is + `qemu-xhci,msix=off,msi=on` (`XHCI_SPARE` in `tests/common/qemu.rs`). + P1 serves usbd when that spare is `qemu-xhci` with its MSI-X left on + and `usbd_drives_the_spare` passes on it. The T14's xHCI has no MSI-X + on its capability walk, whose end is not read + (`issues/every-driver-is-still-in-the-kernel.md`). - QEMU's spare today is `qemu-xhci,msix=off,msi=on`: with MSI-X off and - MSI left `auto` it offers no MSI either, and the claim is refused for - it. Read on the T14 (stage 0, `boot:testcases` readbacks): firmware - owns neither controller. 00:0d.0, 8086:9a13, the Type-C one (port 1 - USB 2.0, ports 2..=5 USB 3.1) and 00:14.0, 8086:a0ed (ports 1..=12 USB + Read on the T14 (stage 0, `boot:testcases` readbacks): firmware owns + neither controller. 00:0d.0, 8086:9a13, the Type-C one (port 1 USB + 2.0, ports 2..=5 USB 3.1) and 00:14.0, 8086:a0ed (ports 1..=12 USB 2.0, 13..=16 USB 3.1) each read `USBLEGSUP 0x01002201` and `USBLEGCTLSTS 0xe0000000` — firmware's semaphore clear, every SMI enable clear, the three status bits latched — and each is armed on @@ -223,11 +231,10 @@ half. since it goes whole here, and its `xhci-xecp-selftest` actuator goes with it. 5. **USB by userland**, with discovery and recovery written once as - straight-line code: step 10, and this exit. **Exit**: no interrupts-off - window longer than a register access, read by stage 6 step 2's - instrument, and keyboard input keeps flowing while a stick misbehaves. - **Partly done**: step 10's stage 1 (#827). **Left**: the rest of step 10, - then this exit. + straight-line code: step 10, and this exit. **Exit**: step 10's, and no + interrupts-off window longer than a register access, read by stage 6 step + 2's instrument. **Partly done**: step 10's stage 1 (#827). **Left**: the + rest of step 10, then this exit. 6. A handler posts its device's `Watch` and ends its interrupt; the thread waiting on that watch does the work, and no step creates a kernel thread. `irq_ring`, the driver list in `drain_irqs` and the idle loop's device @@ -236,30 +243,16 @@ half. windows against the readings step 2's exit puts in the tree, by that exit's rule. Steps 3 and 4 do not land alone: they land with stage 7 step 2 and with usbd. **Partly done**: step 1, and step 2's instrument. - 1. **Interrupts post.** A post is legal in a handler: the watches a handler - posts, and the completions of a ring they complete into, sit behind - interrupts-off locks nothing allocates or frees under, and every other - watch's lock leaves interrupts open. A claimed function's vector, the - IOMMU's refusal and both audio backends post from the handler, and - `irq_ring`'s `UserDev` and `Audio` and their arms in `drain_irqs` go. - The thread is the holder's: netstack's, diskserver's, soundserver's mix - thread, and an `isa` claim's holder. **Exit**: - `handler_post_without_a_pass`, a vector taken on a CPU holding - preemption off, inside a post of its own watch, inside a completion - into a ring polling it, or inside that ring's own watch, posting once - that section lets go and before any pass, red on the base; the watch's - loom models over the new post. **Done** (#634). #660 cut + 1. **Interrupts post.** **Done** (#634). #660 cut its guest test, `handler_post_without_a_pass`; the guest-suite track's stage C owes it - as a T14 row, and the owner's testing ruling (#890) keeps it, since what - it checks is the kernel's alone. + as a T14 row, which #890 leaves standing, since what it checks is the + kernel's alone. 2. **The windows, measured**: the longest interrupts-off and preemption-off windows per CPU, reported beside the IRQ census and fed by each architecture's masking primitives and entries, the number the ARM track's stage 4 owes as well. **Partly done** (#649): the - `mask-windows` kernel and the T14's `mask_windows` row; the instrument - reads back a window of known length on the T14, which is x86 metal, - and the row's judge refuses a boot that reads it back shorter than it - was held or at more than twice that. **Left**, the exit: neither window + `mask-windows` kernel and the T14's `mask_windows` row. **Left**, the + exit: neither window is longer with step 1 than with it reverted on the tree that carries the instrument, the patch posted on #649 being what reverted means. That comparison is the median of at least five interleaved boots an @@ -291,17 +284,24 @@ half. userland, and a dead kernel takes no input, with no emergency way. **Partly done**: step 1. 1. **The panic console takes no input, and an `isa` claim grants a process - exact ports through the TSS I/O permission bitmap and its ISA lines as - records** (`kernel/src/isa.rs`). **Done** (#592). + exact ports and its ISA lines.** **Done** (#592). 2. **ps2server**, the server over that claim, feeding the kernel's keyboard and mouse streams so the merge with USB HID stays where it is; the kernel's driver, its vector, its actuators and the `keyboard_controller` seam deleted. A keyboard claim is refused while no source exists, which the supervisor's order of endowment then decides. - **Exit**: no i8042 code in `kernel/src` (`arch/x86_64/i8042/`, + **Exit**: no i8042 code in the kernel (`arch/x86_64/i8042/`, `arch/x86_64/idt/i8042.rs` and both architectures' - `keyboard_controller` gone), and typing resumes after ps2server is - killed and restarted. **Not started.** + `keyboard_controller` gone from `kernel/src`, and `toyos-ps2`, whose + one kernel user is `kernel/src/arch/x86_64/i8042/mod.rs`, gone from + `kernel/Cargo.toml`); a kernel test this step owes, that a keyboard + claim whose holder dies is handed to the next claimant and the + keyboard stream delivers its transitions and none of the dead + holder's, red on a kernel that keeps the dead holder as the source; + and ps2server's restart mid-packet held by `toyos-ps2`'s + `a_lost_scancode_byte_never_wedges_the_prefix_state` and + `a_stream_truncated_at_any_offset_resyncs_within_two_packets`. + **Not started.** ## Standing From 638c1b0fa1a58542fdb1344358b53bc5b13efd19 Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 01:39:01 +0200 Subject: [PATCH 3/5] The small-kernel track answers review round 2 of #900 Stage 3's exit names, as diskserver's part, the restart test issues/nothing-reopens-a-diskserver-session-over-a-used-page.md owes, red under the mutation that moves the server's stores after its answer: the blockring tests hold the transport and are green with that order reverted. Step 10's /log clause names the same reopen test for usbd's session open, and the owed FAT kill-point test names its negative control, a sync that writes a growing file's directory entry before its chain. The guest-suite track's stage D no longer owes fsd_restart, home_overwrite_reads_back or apps_and_home_are_one_filesystem as metal rows: each claim has the one owner the small-kernel track gives it, and fsd_restart's points at stage 3's module tests. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...-runs-only-what-no-cheaper-tier-reaches.md | 9 ++++---- ...-small-interrupts-post-and-threads-wait.md | 21 +++++++++++++------ 2 files changed, 20 insertions(+), 10 deletions(-) diff --git a/issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md b/issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md index 82d92324378..479845d810c 100644 --- a/issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md +++ b/issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md @@ -115,10 +115,11 @@ and NVMe. Exit: as stage A. protocol, its restart, and its refusals with no controller. - host `nvme_large_device`, `nvme_wide_sector`: a device past 2 TiB and an 8 KiB namespace. - host `fsd_two_data`: two DATA partitions are refused by name. -- metal `fsd_restart`, `fsd_end_at_mount`, `fsd_claim_held`: a file server ended under its - clients is restarted, or refused while its claim is held. -- metal `home_overwrite_reads_back`, `apps_and_home_are_one_filesystem`, `layout_fresh_boot`: - `/home`'s bytes and layout on the disk. +- metal `fsd_end_at_mount`, `fsd_claim_held`: a file server ended at its mount is restarted, + or refused while its claim is held. `fsd_restart`'s claim, a file server ended under its + clients, is held by the module tests stage 3 of + `issues/the-kernel-is-small-interrupts-post-and-threads-wait.md` names. +- metal `layout_fresh_boot`: `/home`'s layout on the disk. - metal `pkg_install_gbae`: `pkg install` lands a package that runs. - metal `partition_claim`: a partition claimed as a device, and its refusals. - metal `kernel_log_file`, `log_partition_layout`, `log_partition_identity`: `/log` on the stick diff --git a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md index b9028e9f75e..78f2b97976f 100644 --- a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md +++ b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md @@ -66,8 +66,12 @@ Beside that order, each waiting on something outside it: death leaves is held by `toyos-blockring`'s `every_request_is_answered_exactly_once`, `what_a_flush_calls_durable_is_on_the_medium` and - `what_was_on_the_wire_at_the_end_is_refused_and_what_was_not_waits`, by - fileserver's + `what_was_on_the_wire_at_the_end_is_refused_and_what_was_not_waits` for + the transport, by the restart test + `issues/nothing-reopens-a-diskserver-session-over-a-used-page.md` owes + for diskserver's part, a session reopened over the page a killed server + left, red under the mutation that moves the server's stores after its + answer, by fileserver's `a_directory_rename_is_whole_under_one_name_wherever_the_server_is_killed` for DATA, and for LOG by the FAT test step 10 owes. **Partly done** (#525, #536): diskserver drives NVMe, and the kernel's NVMe driver is @@ -180,10 +184,15 @@ Beside that order, each waiting on something outside it: a stick whose bulk endpoint never answers leaves a keyboard on the same controller delivering every report, red on a usbd that waits on the stick before it reads the keyboard's ring. `/log`'s, what a killed - usbd had on the wire: stage 3's `toyos-blockring` tests, with one test - this step owes in `system/fileserver/src/fat.rs`: a disk stopped after - each block write in turn of an append and its sync mounts at every - stop and reads back all the last whole sync held. + usbd had on the wire: stage 3's `toyos-blockring` tests for the + transport, and for usbd's session open the reopen test + `issues/nothing-reopens-a-diskserver-session-over-a-used-page.md` + owes, red under the mutation that moves usbd's stores after its + answer; with one test this step owes in + `system/fileserver/src/fat.rs`: a disk stopped after each block write + in turn of an append and its sync mounts at every stop and reads back + all the last whole sync held, red on a sync that writes a growing + file's directory entry before its chain. **Partly done** (#827), its stage 1: the test-only `xhci-leave=:` leaves one controller to a claim, and From 7baabf2c3e58fce1308cf577637d65bb4549c7d0 Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 02:16:13 +0200 Subject: [PATCH 4/5] The small-kernel track answers review round 3 of #900 The guest-suite track points at stage 3 only for the bytes a file server ended under its clients had flushed. Stage 4 records as owed, with the orchestrator as owner, the client's Gone and reconnect as a module test in sdk/toyos/src/fs.rs, red without the generation check, and the supervisor's restart, its RESTARTS bound and its by-name refusal of a held claim as a supervisor module test, red on an unbounded restart. fsd_end_at_mount and fsd_claim_held leave stage D under #890, by the orchestrator's ruling, pointing at that owed test and at partition_claim's row for the kernel's AlreadyExists. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...est-suite-runs-only-what-no-cheaper-tier-reaches.md | 10 ++++++---- ...kernel-is-small-interrupts-post-and-threads-wait.md | 9 +++++++++ 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md b/issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md index 479845d810c..51cd48e6c5b 100644 --- a/issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md +++ b/issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md @@ -115,10 +115,12 @@ and NVMe. Exit: as stage A. protocol, its restart, and its refusals with no controller. - host `nvme_large_device`, `nvme_wide_sector`: a device past 2 TiB and an 8 KiB namespace. - host `fsd_two_data`: two DATA partitions are refused by name. -- metal `fsd_end_at_mount`, `fsd_claim_held`: a file server ended at its mount is restarted, - or refused while its claim is held. `fsd_restart`'s claim, a file server ended under its - clients, is held by the module tests stage 3 of - `issues/the-kernel-is-small-interrupts-post-and-threads-wait.md` names. +- `fsd_restart`, `fsd_end_at_mount` and `fsd_claim_held` are out under #890. Of + `issues/the-kernel-is-small-interrupts-post-and-threads-wait.md`, stage 3's module tests hold + the bytes a file server ended under its clients had flushed, and stage 4 owes the client's + `Gone` and reconnect, and the supervisor's restart, its bound and its refusal by name of a + restart whose claim is held, as module tests. The kernel's `AlreadyExists` on a held partition + is `partition_claim`'s. - metal `layout_fresh_boot`: `/home`'s layout on the disk. - metal `pkg_install_gbae`: `pkg install` lands a package that runs. - metal `partition_claim`: a partition claimed as a device, and its refusals. diff --git a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md index 78f2b97976f..8f9799bb059 100644 --- a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md +++ b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md @@ -92,6 +92,15 @@ Beside that order, each waiting on something outside it: program. Ctrl+Alt+D is removed (owner, 2026-10-10: "i dont want it. i want it removed."; done, #824). No swap, declared. `SYS_DEVICE_DMA_MAP` for zero-copy block I/O. FAT32 on `/log` in the installed product is deferred. + **Owed**, the orchestrator's to dispatch, for the restart beyond the + bytes stage 3 holds: a module test in `sdk/toyos/src/fs.rs` that a call + on an id from an ended connection answers `Gone` and a path call + reconnects once, red with the generation check removed; and a module + test in `system/supervisor` that an ended role is started again until + `toyos_manifest::RESTARTS` ends inside the window and is then closed with + its clients answered `Gone`, red on a supervisor that restarts without + bound, and that a restart whose partition claim another process holds is + refused by name. **Partly done**: steps 1, 2, 3, 6 and 7 are done, and 4, 5, 8, 9 and 10 partly. From 0abff654ce0c3933b5ddde20c48594df02fc8a4c Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 02:26:00 +0200 Subject: [PATCH 5/5] The small-kernel track names the owed supervisor test's mutation for the refusal by name Review round 4 of #900 noted the owed supervisor module test named a mutation only for its RESTARTS clause. The refusal-by-name clause now names its own: red on a supervisor that starts the role again while its claim is held, or that drops the name from the AlreadyExists refusal. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- .../the-kernel-is-small-interrupts-post-and-threads-wait.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md index 8f9799bb059..f5309e30184 100644 --- a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md +++ b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md @@ -100,7 +100,9 @@ Beside that order, each waiting on something outside it: `toyos_manifest::RESTARTS` ends inside the window and is then closed with its clients answered `Gone`, red on a supervisor that restarts without bound, and that a restart whose partition claim another process holds is - refused by name. + refused by name, red on a supervisor that starts the role again while its + claim is held or that drops the name from the `AlreadyExists` refusal + (`refused` in `system/supervisor/src/main.rs`). **Partly done**: steps 1, 2, 3, 6 and 7 are done, and 4, 5, 8, 9 and 10 partly.