From 26609b4cd25d1ccac4c07b49a00fec32db3a9091 Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 14 Oct 2026 13:13:23 +0200 Subject: [PATCH 1/6] A device function is claimed by its arrival, and a drives row has one process per function Stage 1.2 of issues/a-device-function-is-driven-from-its-arrival-to-its-end.md. The kernel no longer resolves a vendor:device id to a function. Each inventory `Pci` record carries an `Arrival`, the function's number for the boot, and `DeviceRequest::Function(arrival)` names it; deciding which function a row asks for is userland's. `DeviceRequest::Pci`, `toyos_pci::claimed` and its `Unclaimable` go. `ClaimError::Ambiguous` stays only where it is still true, a partition GUID carried twice. `lib/manifest` owns the PCI matcher grammar (`pci:CC:SS:PP` and `pci:CC:SS:PP/VVVV:DDDD`, `device::PciMatch`). Every matcher carries its class, so two entries one function can answer are refused when the manifest renders, naming both rows and entries; that refusal also takes over `one_claimant_per_device`'s duplicate class, `isa` and `part:` names. A `drives` row is one process per function its matcher names and the kernel does not drive: it serves nothing, holds no `devices`, has no `roles`, is in no `[boot] start`, and reaches its consumer through `receives`. The supervisor's lifecycle decisions are a pure crate, `toyos-supervisor` (`system/supervisor/lifecycle`), host-tested: restart policy and its budget, the claims a restart is owed, the one function a single-process `devices` matcher names (two refuse that row's start, naming both), and which functions a `drives` row has a process on. A refused start leaves its function alone until its inventory record changes, and every start of a function after its first spends its budget, so a reading that changed nothing starts nothing. The supervisor watches each child's `Process` handle on its loop's own poller and on the file worker's wait; the waiter threads, `Kept`'s mutex, the wake pipe and `Program::supervised` go. vdagent is a `drives` row, and its "no virtio-serial function" exit goes with it. Every `pci:` entry in the images gains its class, read from QEMU 11.1.1's configuration space. The test images whose jobs claim a function hold `inventory`, and the jobs find a function's arrival through one helper. `a_drives_row_holds_one_process_per_function` boots a q35 with two `edu` (`Profile::HeadlessTwoEdu`, `tests/images/drivescase.toml`): one `pciholder` per function, each saying its own, both still held when the job judges them. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- Cargo.lock | 18 + Cargo.toml | 2 + build/build.rs | 86 +-- images/console.toml | 2 +- images/system.toml | 22 +- kernel/pci/src/choose.rs | 60 +- kernel/pci/src/lib.rs | 2 +- kernel/src/device.rs | 20 +- kernel/src/drivers/pci.rs | 7 +- kernel/src/pcidev/mod.rs | 71 +-- kernel/src/syscall/device.rs | 2 +- lib/inspect/src/dev.rs | 4 +- lib/manifest/src/device.rs | 193 ++++++ lib/manifest/src/lib.rs | 225 ++++--- sdk/abi/src/inventory.rs | 9 + sdk/abi/src/syscall.rs | 120 ++-- sdk/toyos/src/endow.rs | 24 +- sdk/toyos/src/syscap.rs | 19 +- system/netstack/Cargo.toml | 2 + system/netstack/src/main.rs | 28 +- system/soundserver/Cargo.toml | 2 + system/soundserver/src/main.rs | 2 +- system/soundserver/src/virtio.rs | 10 +- system/supervisor/Cargo.toml | 3 + system/supervisor/lifecycle/Cargo.toml | 16 + system/supervisor/lifecycle/src/claims.rs | 108 ++++ system/supervisor/lifecycle/src/drives.rs | 246 +++++++ system/supervisor/lifecycle/src/lib.rs | 10 + system/supervisor/lifecycle/src/restart.rs | 104 +++ system/supervisor/src/main.rs | 599 ++++++++++-------- system/vdagent/src/main.rs | 16 +- tests/common/iommu.rs | 6 +- tests/common/qemu.rs | 17 +- tests/guest/pciholder/Cargo.toml | 9 + tests/guest/pciholder/src/main.rs | 23 + .../guest/rust-tests/src/bin/bar_map_again.rs | 5 +- .../rust-tests/src/bin/claim_bar_fault.rs | 5 +- tests/guest/rust-tests/src/bin/claim_lpi.rs | 5 +- .../rust-tests/src/bin/claim_unrouted.rs | 7 +- .../guest/rust-tests/src/bin/drives_judged.rs | 43 ++ .../rust-tests/src/bin/end_returns_claims.rs | 12 +- tests/guest/rust-tests/src/bin/i219_msi.rs | 5 +- .../src/bin/lent_device_memory_refused.rs | 5 +- tests/guest/rust-tests/src/bin/pci_reclaim.rs | 5 +- tests/guest/rust-tests/src/bin/usbd_spare.rs | 10 +- tests/guest/rust-tests/src/function.rs | 27 + tests/images/acpicase.toml | 2 +- tests/images/clipboardcase.toml | 6 +- tests/images/drivescase.toml | 21 + tests/images/jobcase.toml | 2 +- tests/images/latencycase.toml | 2 +- tests/images/metalcase.toml | 6 +- tests/images/metaldevicecase.toml | 2 +- tests/images/netcase.toml | 8 +- tests/images/proctreecase.toml | 2 +- tests/images/testcases.toml | 8 +- tests/images/virtclaimcase.toml | 8 +- tests/toyos.rs | 48 ++ 58 files changed, 1648 insertions(+), 683 deletions(-) create mode 100644 lib/manifest/src/device.rs create mode 100644 system/supervisor/lifecycle/Cargo.toml create mode 100644 system/supervisor/lifecycle/src/claims.rs create mode 100644 system/supervisor/lifecycle/src/drives.rs create mode 100644 system/supervisor/lifecycle/src/lib.rs create mode 100644 system/supervisor/lifecycle/src/restart.rs create mode 100644 tests/guest/pciholder/Cargo.toml create mode 100644 tests/guest/pciholder/src/main.rs create mode 100644 tests/guest/rust-tests/src/bin/drives_judged.rs create mode 100644 tests/guest/rust-tests/src/function.rs create mode 100644 tests/images/drivescase.toml diff --git a/Cargo.lock b/Cargo.lock index 4d7d4192dc8..5d9741ba61e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2107,6 +2107,7 @@ dependencies = [ "toyos-dns", "toyos-i219", "toyos-inspect", + "toyos-manifest", "toyos-mdns", "toyos-net-ip", "toyos-net-node", @@ -2701,6 +2702,13 @@ dependencies = [ "thiserror 1.0.69", ] +[[package]] +name = "pciholder" +version = "0.1.0" +dependencies = [ + "toyos", +] + [[package]] name = "pem" version = "4.0.0" @@ -3690,6 +3698,7 @@ dependencies = [ "toyos-abi", "toyos-hda", "toyos-inspect", + "toyos-manifest", "toyos-mixer", "toyos-pci-claim", "toyos-resample", @@ -3775,6 +3784,7 @@ dependencies = [ "toyos-manifest", "toyos-osrelease", "toyos-quiesce", + "toyos-supervisor", "toyos-swap", "toyos-update", "toyos-wallclock", @@ -4478,6 +4488,14 @@ dependencies = [ name = "toyos-std-copies" version = "0.1.0" +[[package]] +name = "toyos-supervisor" +version = "0.1.0" +dependencies = [ + "toyos-abi", + "toyos-manifest", +] + [[package]] name = "toyos-swap" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index 266e2e06747..3195f938f16 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -123,6 +123,7 @@ members = [ "system/soundserver/virtio-sound", "system/sshserver", "system/supervisor", + "system/supervisor/lifecycle", "system/swap", "system/symbolize", "system/terminal", @@ -136,6 +137,7 @@ members = [ "system/vdagent/bridge", "tests/guest/kernelprobe", "tests/guest/metalprobe", + "tests/guest/pciholder", "tests/guest/test-runner", "tests/metal", ] diff --git a/build/build.rs b/build/build.rs index 824956a3d63..16cf596e801 100644 --- a/build/build.rs +++ b/build/build.rs @@ -170,8 +170,12 @@ struct ProgramConfig { provides: Vec, /// Names in this program's namespace, each a *connector*. receives: Vec, - /// Device classes the supervisor mints a claim for and endows. + /// Claims the supervisor mints for this row's one process and endows + /// (`toyos_manifest::Program::devices`). devices: Vec, + /// PCI matchers: one process of this row per function one names + /// (`toyos_manifest::Program::drives`). + drives: Vec, /// Rights on the `SysCap` duplicate the supervisor endows this program, by the names /// `toyos_manifest::syscap_rights` takes. A handful of rows in the whole /// tree declare one. @@ -738,6 +742,7 @@ fn render_manifest(config: &SystemConfig) -> Vec { provides: cfg.provides.clone(), receives: cfg.receives.clone(), devices: cfg.devices.clone(), + drives: cfg.drives.clone(), syscap: cfg.syscap.clone(), slots: cfg.slots, service: cfg.service, @@ -3044,6 +3049,7 @@ mod tests { "images/console.toml", "tests/images/acpicase.toml", "tests/images/clipboardcase.toml", + "tests/images/drivescase.toml", "tests/images/hostedclangcase.toml", "tests/images/jobcase.toml", "tests/images/latencycase.toml", @@ -3242,52 +3248,14 @@ mod tests { assert!(provides_disjoint_from_serves(&bad).is_err()); } - /// The supervisor mints one claim per device, so a config naming one twice starts a - /// program with a hole where its claim should be. - fn one_claimant_per_device(cfg: &SystemConfig) -> Result<(), String> { - let mut seen: BTreeMap<&str, &str> = BTreeMap::new(); - for (name, prog) in &cfg.programs { - for d in &prog.devices { - if let Some(prev) = seen.insert(d, name) { - return Err(format!( - "device `{d}` is claimed by both `{prev}` and `{name}`; the second \ - claim is refused at boot" - )); - } - } - } - Ok(()) - } - - /// Not the capability boundary — `kernel/src/pcidev`'s slot reservation is, - /// and this compares `system.toml` strings. - #[test] - fn every_device_class_has_at_most_one_claimant() { - for cfg in ALL_CONFIGS { - one_claimant_per_device(&load(cfg)).unwrap_or_else(|e| panic!("{cfg}: {e}")); - } - let bad: SystemConfig = toml::from_str( - "[programs.a]\ndevices = [\"framebuffer\"]\n\ - [programs.b]\ndevices = [\"framebuffer\"]\n", - ) - .unwrap(); - assert!(one_claimant_per_device(&bad).is_err()); - } - - /// A device name the ABI does not know renders fine and leaves the supervisor with a - /// `devices` entry it cannot mint — a dead machine for a typo, where this is - /// a red in milliseconds. Same for a `syscap` right. - /// - /// The ABI's own parser, not a copy of it: a `pci::` entry - /// names a function and a class name names a class, and this is the same - /// `DeviceRequest::parse` the supervisor and the kernel read the entry with. + /// A `syscap` right the ABI does not know renders fine and leaves the + /// supervisor with a row it cannot endow — a dead machine for a typo, + /// where this is a red in milliseconds. A device entry that names nothing, + /// and two one device answers, are the manifest's own render refusals + /// (`toyos_manifest::render`), which `every_config_renders_its_manifest` + /// holds every config to. fn names_only_real_capabilities(cfg: &SystemConfig) -> Result<(), String> { for (name, prog) in &cfg.programs { - for device in &prog.devices { - if toyos_manifest::DeviceRequest::parse(device).is_none() { - return Err(format!("`{name}` names device `{device}`, which is not one")); - } - } toyos_manifest::syscap_rights(&prog.syscap) .map_err(|e| format!("`{name}`: {e}"))?; } @@ -3299,23 +3267,29 @@ mod tests { for cfg in ALL_CONFIGS { names_only_real_capabilities(&load(cfg)).unwrap_or_else(|e| panic!("{cfg}: {e}")); } - let bad_class: SystemConfig = - toml::from_str("[programs.a]\ndevices = [\"gpu\"]\n").unwrap(); - assert!(names_only_real_capabilities(&bad_class).is_err()); - // A PCI entry that names no function is the same defect one level down, - // and the one a hand-written config is most likely to make. - let bad_function: SystemConfig = - toml::from_str("[programs.a]\ndevices = [\"pci:1af4\"]\n").unwrap(); - assert!(names_only_real_capabilities(&bad_function).is_err()); let bad_right: SystemConfig = toml::from_str("[programs.a]\nsyscap = [\"root\"]\n").unwrap(); assert!(names_only_real_capabilities(&bad_right).is_err()); } + /// Two rows naming one function are refused where the image is built, by + /// both rows' names: the manifest's refusal reaches the build. + #[test] + fn two_rows_on_one_function_are_a_build_refusal_by_name() { + let config: SystemConfig = toml::from_str( + "[programs.netstack]\ndevices = [\"pci:02:00:00/1af4:1041\"]\n\ + [programs.nic]\ndrives = [\"pci:02:00:00\"]\n", + ) + .unwrap(); + let refused = std::panic::catch_unwind(|| render_manifest(&config)).expect_err("rendered"); + let said = refused.downcast_ref::().expect("a formatted panic"); + assert!(said.contains("netstack") && said.contains("nic") && said.contains("Overlap"), "{said}"); + } + fn claims_no_device(cfg: &SystemConfig) -> Result<(), String> { for (name, prog) in &cfg.programs { - if !prog.devices.is_empty() { - return Err(format!("program `{name}` claims {:?}", prog.devices)); + if !prog.devices.is_empty() || !prog.drives.is_empty() { + return Err(format!("program `{name}` claims {:?} and drives {:?}", prog.devices, prog.drives)); } } Ok(()) @@ -3332,6 +3306,8 @@ mod tests { let bad: SystemConfig = toml::from_str("[programs.x]\ndevices = [\"framebuffer\"]\n").unwrap(); assert!(claims_no_device(&bad).is_err()); + let driving: SystemConfig = toml::from_str("[programs.x]\ndrives = [\"pci:03:00:00\"]\n").unwrap(); + assert!(claims_no_device(&driving).is_err()); } /// `[boot] start` names program keys, so a typo is a build error rather than diff --git a/images/console.toml b/images/console.toml index d5851a42010..5c28649cb0d 100644 --- a/images/console.toml +++ b/images/console.toml @@ -99,7 +99,7 @@ syscap = ["roster"] service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/images/system.toml b/images/system.toml index 740113a1c60..9fae23d7246 100644 --- a/images/system.toml +++ b/images/system.toml @@ -19,7 +19,7 @@ assets = ["assets"] # while the compositor had no way to ask for a program's *declared* authority — # the launcher gives it one, and starting a service from `[boot] start` is not # the thing that needed fixing. -start = ["logkeeper", "diskserver", "fileserver", "compositor", "soundserver", "netstack", "filepicker", "acpiserver", "vdagent", "ps2server"] +start = ["logkeeper", "diskserver", "fileserver", "compositor", "soundserver", "netstack", "filepicker", "acpiserver", "ps2server"] # What every program launched out of `/apps` holds, whatever its own directory # says. A package is unpacked by `/system/bin/pkg` into a writable directory, so @@ -98,25 +98,25 @@ devices = ["isa:0060,0064:1,12"] [programs.soundserver] service = true serves = ["soundserver", "volume"] -devices = ["hda-audio", "pci:1af4:1059"] +devices = ["hda-audio", "pci:04:01:00/1af4:1059"] syscap = ["rt"] # netstack holds the NIC's PCI function and drives it: the virtqueues, the register # window and the interrupt are its own, and the kernel keeps only the claim. -# Named by vendor and device rather than by slot, so one row finds the card +# Named by class, vendor and device rather than by slot, so one row finds the card # wherever firmware put it. [programs.netstack] service = true serves = ["netstack"] -devices = ["pci:1af4:1041"] +devices = ["pci:02:00:00/1af4:1041"] -# The shared clipboard with QEMU's host: the virtio-serial function its -# vdagent chardev is on, claimed by vendor and device among the functions the -# kernel does not drive (the kernel's console is `1af4:1043` too), and the -# compositor's clipboard port. A machine with no such function, the T14 among -# them, leaves it to say so and exit. +# The shared clipboard with QEMU's host: one process per virtio-serial +# function the kernel does not drive (the kernel's console is `1af4:1043` too), +# holding the function its vdagent chardev is on, and the compositor's +# clipboard port. A machine with no such function, the T14 among them, starts +# none. [programs.vdagent] -devices = ["pci:1af4:1043"] +drives = ["pci:07:80:00/1af4:1043"] receives = ["clipboard"] # Every launch it makes is in a login session, the one `update` and `swap` @@ -255,7 +255,7 @@ starts = ["toybox", "shell", "terminal", "doom", "snake", "calc", "editor", "pai service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/kernel/pci/src/choose.rs b/kernel/pci/src/choose.rs index 19d21a4cdf5..4305649f1ee 100644 --- a/kernel/pci/src/choose.rs +++ b/kernel/pci/src/choose.rs @@ -1,11 +1,10 @@ -//! Which of the functions one `vendor:device` names is the one asked for: the -//! decision alone, over what the kernel read of each. +//! Which of the functions one `vendor:device` names the kernel's console +//! drives: the decision alone, over what the kernel read of each. //! //! **Two functions can carry one ID and be two different devices.** QEMU's //! virtio-serial is `1af4:1043` whether it carries the kernel's single-port -//! console or a multiport channel a process drives, so the kernel's console -//! driver picks by what the device offers ([`console`]), and a claim picks -//! among the functions the kernel does not drive ([`claimed`]). +//! console or a multiport channel a process drives, so the console driver +//! picks by what the device offers. /// Virtio 1.2 §5.3.3: the console device's `VIRTIO_CONSOLE_F_MULTIPORT`. pub const VIRTIO_CONSOLE_F_MULTIPORT: u64 = 1 << 1; @@ -21,41 +20,6 @@ pub fn console(features: &[u64]) -> Option { features.iter().position(|offered| offered & VIRTIO_CONSOLE_F_MULTIPORT == 0) } -/// Why a claim's `vendor:device` names no function to hand over. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -pub enum Unclaimable { - /// No function carries it. - Absent, - /// Two or more functions the kernel does not drive carry it: the name asks - /// for one of them and cannot say which. - Ambiguous, - /// Every function that carries it is the kernel's own. - KernelDriven, -} - -/// The function a claim resolves to, among those that carry its -/// `vendor:device` in enumeration order, each `true` where the kernel drives -/// it: the one function the kernel does not drive. -pub fn claimed(driven: impl IntoIterator) -> Result { - let mut found = None; - let mut any = false; - for (at, driven) in driven.into_iter().enumerate() { - any = true; - if driven { - continue; - } - if found.is_some() { - return Err(Unclaimable::Ambiguous); - } - found = Some(at); - } - match (found, any) { - (Some(at), _) => Ok(at), - (None, true) => Err(Unclaimable::KernelDriven), - (None, false) => Err(Unclaimable::Absent), - } -} - #[cfg(test)] mod tests { use super::*; @@ -75,20 +39,4 @@ mod tests { assert_eq!(console(&[MULTI, MULTI]), None); assert_eq!(console(&[]), None); } - - #[test] - fn one_undriven_match_among_driven_ones_is_claimed() { - assert_eq!(claimed([true, false, true]), Ok(1)); - } - - #[test] - fn two_undriven_matches_are_ambiguous() { - assert_eq!(claimed([false, true, false]), Err(Unclaimable::Ambiguous)); - } - - #[test] - fn only_driven_matches_are_kernel_driven() { - assert_eq!(claimed([true, true]), Err(Unclaimable::KernelDriven)); - assert_eq!(claimed([]), Err(Unclaimable::Absent)); - } } diff --git a/kernel/pci/src/lib.rs b/kernel/pci/src/lib.rs index 8d28427a5c3..5ce5a208d04 100644 --- a/kernel/pci/src/lib.rs +++ b/kernel/pci/src/lib.rs @@ -40,4 +40,4 @@ pub mod placement; pub mod pm; pub mod probe; -pub use choose::{claimed, console, Unclaimable}; +pub use choose::console; diff --git a/kernel/src/device.rs b/kernel/src/device.rs index 66c71e75596..33358c1ad07 100644 --- a/kernel/src/device.rs +++ b/kernel/src/device.rs @@ -151,13 +151,11 @@ pub enum ClaimError { Owned, /// This machine has no such device — no driver ever registered one. Absent, - /// The name matches more than one function this kernel does not drive. - /// Refused rather than resolved to the first: a config asking for one of - /// two identical cards cannot say which, and picking would be the kernel - /// deciding. + /// The partition GUID is carried twice. Refused rather than resolved to + /// the first: picking would be the kernel deciding. Ambiguous, - /// A driver in this kernel already binds every function the name matches; - /// two drivers on one device is not something a claim may create. + /// A driver in this kernel already binds the device; two drivers on one + /// device is not something a claim may create. KernelDriven, /// This machine has no interrupt left to give the function: every vector, /// remapping entry or LPI a claim spends is held. @@ -174,8 +172,8 @@ pub enum ClaimError { /// failure after `acquire` cannot leave a device held by nobody. /// /// `selector` says *which* device where the class alone does not — a PCI -/// function's vendor and device id in its first word, a partition's GUID in -/// both — and the syscall has refused a word the class does not read. +/// function's arrival in its first word, a partition's GUID in both — and the +/// syscall has refused a word the class does not read. pub fn try_claim(class: DeviceType, selector: [u64; 2]) -> Result, ClaimError> { // Availability is checked before acquiring, so an absent device reports `Absent`, not `Owned`. match class { @@ -197,12 +195,12 @@ pub fn try_claim(class: DeviceType, selector: [u64; 2]) -> Result { - let id = toyos_abi::syscall::PciId::from_wire(selector[0]) - .ok_or(ClaimError::Absent)?; + // A number past a `u32` is an arrival no function had. + let arrival = u32::try_from(selector[0]).map_err(|_| ClaimError::Absent)?; // The function's own guard, taken inside: a PCI claim's exclusivity is // per function rather than per class, so there is no flag here to // acquire first. - let (info, claim) = crate::pcidev::claim(id)?; + let (info, claim) = crate::pcidev::claim(toyos_abi::inventory::Arrival(arrival))?; Ok(DeviceClaim::new(class, DeviceInfo::PciFunction(info), claim)) } DeviceType::Partition => { diff --git a/kernel/src/drivers/pci.rs b/kernel/src/drivers/pci.rs index 750fd54ff6f..3e8e7e51750 100644 --- a/kernel/src/drivers/pci.rs +++ b/kernel/src/drivers/pci.rs @@ -462,8 +462,13 @@ impl PciDevice { /// What this function at `at` says it is, for the inventory; driven by /// nobody until [`crate::pcidev::inventory`] says otherwise. - pub fn identity(&self, at: toyos_abi::inventory::PciAddr) -> toyos_abi::inventory::Pci { + pub fn identity( + &self, + arrival: toyos_abi::inventory::Arrival, + at: toyos_abi::inventory::PciAddr, + ) -> toyos_abi::inventory::Pci { toyos_abi::inventory::Pci { + arrival, at, vendor: self.vendor_id(), device: self.device_id(), diff --git a/kernel/src/pcidev/mod.rs b/kernel/src/pcidev/mod.rs index 06c541f6116..1cc8100cd3f 100644 --- a/kernel/src/pcidev/mod.rs +++ b/kernel/src/pcidev/mod.rs @@ -123,6 +123,7 @@ use core::sync::atomic::{AtomicBool, AtomicPtr, Ordering}; use record::Interrupt; use toyos_abi::boot::{MemoryMapEntry, RootBridgeWindow}; +use toyos_abi::inventory::Arrival; use toyos_abi::pci::{DeviceIrqRecord, PciFunctionInfo, BARS}; use toyos_abi::syscall::{PciId, RegWidth, SyscallError}; use toyos_dma::Register; @@ -173,9 +174,8 @@ struct Function { /// /// **This is where a function's exclusivity is decided.** /// `Claim::acquire`'s per-class flag does not answer for a class that - /// names several devices, `build/build.rs`'s `one_claimant_per_device` - /// compares `system.toml` strings, and a host-side gate is not the - /// capability boundary. + /// names several devices, and the manifest's refusal of two matchers on + /// one function is a host-side gate, not the capability boundary. held: AtomicBool, irq: Interrupt, /// What its holder's poll waits on: two processes each driving a function @@ -200,7 +200,8 @@ impl core::fmt::Display for Function { /// Exactly the functions [`publish`] took, in [`Machine::functions`]' order, /// leaked once: null until then and never written again, so a handler reaches -/// a function's record with no lock. +/// a function's record with no lock. A function's place here is its +/// [`Arrival`]'s ([`arrival_of`]), which is therefore never another's. static FUNCTIONS: AtomicPtr<&'static [Function]> = AtomicPtr::new(core::ptr::null_mut()); fn functions() -> &'static [Function] { @@ -213,6 +214,12 @@ fn functions() -> &'static [Function] { unsafe { *published } } +/// The arrival of function `number`, counted from 1 so a zeroed record names +/// none. +fn arrival_of(number: usize) -> Arrival { + Arrival(u32::try_from(number + 1).expect("pcidev: fewer functions than a u32 counts")) +} + /// Function `number`, which a claim or an interrupt named. fn function(number: usize) -> &'static Function { functions().get(number).unwrap_or_else(|| panic!("pcidev: function {number} was never enumerated")) @@ -599,8 +606,11 @@ pub fn publish(devices: &[PciDevice], segment: u16, maps: &[MemoryMapEntry], fir assert!(first.is_ok(), "pcidev: the functions were published twice"); let mut machine = MACHINE.lock(); machine.functions = devices.to_vec(); - machine.identities = - devices.iter().map(|d| d.identity(addr_of(segment, requester(d)))).collect(); + machine.identities = devices + .iter() + .enumerate() + .map(|(number, d)| d.identity(arrival_of(number), addr_of(segment, requester(d)))) + .collect(); machine.segment = segment; machine.decoded = decoded; machine.firmware = firmware.to_vec(); @@ -759,45 +769,20 @@ impl core::fmt::Display for Refusal { } } -/// Claim `id`, bring the function up to the point a driver takes over, and -/// answer what that driver needs to know. -pub fn claim(id: PciId) -> Result<(PciFunctionInfo, Claim), ClaimError> { - // Among the functions this kernel does not drive (`toyos_pci::claimed`): - // the console's virtio-serial and a process's carry one ID. - let (matches, resolved) = { +/// Claim the function that arrived as `arrival`, bring it up to the point a +/// driver takes over, and answer what that driver needs to know. +pub fn claim(arrival: Arrival) -> Result<(PciFunctionInfo, Claim), ClaimError> { + let (number, pci, id, driven) = { let machine = MACHINE.lock(); - let matches: Vec<(usize, PciDevice)> = - machine.functions.iter().copied().enumerate().filter(|(_, d)| d.is_id(id.vendor, id.device)).collect(); - let resolved = - toyos_pci::claimed(matches.iter().map(|(_, d)| machine.kernel_driven.contains(&requester(d)))); - (matches, resolved) - }; - let (number, pci) = match resolved { - Ok(at) => matches[at], - Err(toyos_pci::Unclaimable::Absent) => return Err(ClaimError::Absent), - Err(toyos_pci::Unclaimable::Ambiguous) => { - // Refused by name rather than resolved to the first match: a - // config that names a card this machine has two of is asking - // for one of them and cannot say which. - log!( - "pcidev: {:04x}:{:04x} names more than one function this kernel does not drive", - id.vendor, - id.device - ); - return Err(ClaimError::Ambiguous); - } - Err(toyos_pci::Unclaimable::KernelDriven) => { - for (_, pci) in &matches { - log!( - "pcidev: PCI {:02x}:{:02x}.{} is driven by this kernel and cannot be claimed", - pci.bus, - pci.dev, - pci.func - ); - } - return Err(ClaimError::KernelDriven); - } + let number = machine.identities.iter().position(|pci| pci.arrival == arrival).ok_or(ClaimError::Absent)?; + let (pci, identity) = (machine.functions[number], &machine.identities[number]); + let driven = machine.kernel_driven.contains(&requester(&pci)); + (number, pci, PciId { vendor: identity.vendor, device: identity.device }, driven) }; + if driven { + log!("pcidev: PCI {:02x}:{:02x}.{} is driven by this kernel and cannot be claimed", pci.bus, pci.dev, pci.func); + return Err(ClaimError::KernelDriven); + } // The test and the take are one exchange: two claims arriving together // must not both find the function unheld. diff --git a/kernel/src/syscall/device.rs b/kernel/src/syscall/device.rs index fa5ed5a203c..6612e2ae3be 100644 --- a/kernel/src/syscall/device.rs +++ b/kernel/src/syscall/device.rs @@ -184,7 +184,7 @@ pub(super) fn sys_acpi(ctx: &SyscallContext, handle: RawHandle, op: u64, at: u64 /// Mints a device claim, gated on a `SysCap` carrying [`Rights::DEVICE`]. /// /// `selector` says which device where the class alone does not — a PCI -/// function's vendor and device id, a partition's GUID — and a word the class +/// function's arrival, a partition's GUID — and a word the class /// does not read is refused, never dropped: whoever set it meant a device this /// claim would not name. pub(super) fn sys_device_claim(syscap: RawHandle, class: u64, selector: [u64; 2]) -> u64 { diff --git a/lib/inspect/src/dev.rs b/lib/inspect/src/dev.rs index f16b335221c..6a8249f8673 100644 --- a/lib/inspect/src/dev.rs +++ b/lib/inspect/src/dev.rs @@ -208,7 +208,7 @@ mod tests { use crate::path::check_path; use crate::Selector; use alloc::vec::Vec; - use toyos_abi::inventory::{Block, Claim, Partition, Pci, Usb, NAME_BYTES}; + use toyos_abi::inventory::{Arrival, Block, Claim, Partition, Pci, Usb, NAME_BYTES}; use toyos_abi::syscall::DeviceType; fn holder(pid: u32, s: &str) -> Holder { @@ -222,7 +222,7 @@ mod tests { const XHCI: PciAddr = PciAddr { segment: 0, bus: 0, dev: 4, func: 0 }; fn pci(at: PciAddr, driven: Driven) -> Record { - Record::Pci(Pci { at, vendor: 0x8086, device: 0x15fc, class: 2, subclass: 0, prog_if: 0, driven }) + Record::Pci(Pci { arrival: Arrival(1), at, vendor: 0x8086, device: 0x15fc, class: 2, subclass: 0, prog_if: 0, driven }) } fn part(device: u32, index: u32, unique: u8, state: PartState) -> Record { diff --git a/lib/manifest/src/device.rs b/lib/manifest/src/device.rs new file mode 100644 index 00000000000..17ee978c805 --- /dev/null +++ b/lib/manifest/src/device.rs @@ -0,0 +1,193 @@ +//! What a `devices` or `drives` entry names. +//! +//! A class, a partition or an ISA set is the ABI's own spelling +//! ([`DeviceRequest::parse`]). A PCI function is matched here, by its class +//! code and optionally its card: `pci:CC:SS:PP` or `pci:CC:SS:PP/VVVV:DDDD`, +//! lowercase hex. **Every matcher carries its class**, so whether two entries +//! can name one function is decided from their text alone ([`Entry::overlaps`]), +//! and a claim names the function a matcher found by its arrival. + +use core::fmt; + +use toyos_abi::inventory::Pci; +use toyos_abi::syscall::{DeviceRequest, PciId}; + +/// One entry of a row's `devices` or `drives`. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub enum Entry { + /// A class, a partition or an ISA set: one device by its own name. + Fixed(DeviceRequest), + Pci(PciMatch), +} + +impl Entry { + pub fn parse(text: &str) -> Option { + match text.strip_prefix(PciMatch::PREFIX) { + Some(_) => PciMatch::parse(text).map(Self::Pci), + None => DeviceRequest::parse(text).map(Self::Fixed), + } + } + + /// Whether one device can answer both: the same fixed name, or two + /// matchers of one class whose cards do not differ. + pub fn overlaps(&self, other: &Self) -> bool { + match (self, other) { + (Self::Fixed(a), Self::Fixed(b)) => a == b, + (Self::Pci(a), Self::Pci(b)) => a.class == b.class && (a.id.is_none() || b.id.is_none() || a.id == b.id), + _ => false, + } + } +} + +/// Which PCI functions an entry names: every function of a class code, or of +/// one card within it. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub struct PciMatch { + /// Base class, subclass and programming interface. + pub class: [u8; 3], + pub id: Option, +} + +impl PciMatch { + pub const PREFIX: &'static str = "pci:"; + + /// Every function of class `class`, of card `id` where it names one. + pub const fn new(class: [u8; 3], id: Option) -> Self { + Self { class, id } + } + + pub fn parse(text: &str) -> Option { + let rest = text.strip_prefix(Self::PREFIX)?; + let (code, card) = match rest.split_once('/') { + Some((code, card)) => (code, Some(PciId::parse(card)?)), + None => (rest, None), + }; + let mut class = [0u8; 3]; + let mut fields = code.split(':'); + for byte in &mut class { + *byte = hex8(fields.next()?)?; + } + if fields.next().is_some() { + return None; + } + Some(Self { class, id: card }) + } + + pub fn matches(&self, pci: &Pci) -> bool { + [pci.class, pci.subclass, pci.prog_if] == self.class + && self.id.is_none_or(|id| id.vendor == pci.vendor && id.device == pci.device) + } +} + +impl fmt::Display for PciMatch { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let [class, subclass, prog_if] = self.class; + write!(f, "{}{class:02x}:{subclass:02x}:{prog_if:02x}", Self::PREFIX)?; + match self.id { + Some(id) => write!(f, "/{:04x}:{:04x}", id.vendor, id.device), + None => Ok(()), + } + } +} + +/// Exactly two lowercase hex digits: one spelling per value, because the +/// label a claim is endowed under is the entry's text. +fn hex8(text: &str) -> Option { + let lowercase = text.bytes().all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)); + if text.len() != 2 || !lowercase { + return None; + } + u8::from_str_radix(text, 16).ok() +} + +#[cfg(test)] +mod tests { + use super::*; + use toyos_abi::inventory::{Arrival, Driven, PciAddr}; + use toyos_abi::syscall::DeviceType; + + const NVME: PciMatch = PciMatch::new([0x01, 0x08, 0x02], Some(PciId { vendor: 0x1b36, device: 0x0010 })); + + fn function(class: [u8; 3], vendor: u16, device: u16) -> Pci { + Pci { + arrival: Arrival(1), + at: PciAddr { segment: 0, bus: 0, dev: 3, func: 0 }, + vendor, + device, + class: class[0], + subclass: class[1], + prog_if: class[2], + driven: Driven::Free, + } + } + + #[test] + fn a_matcher_writes_back_the_text_it_parsed() { + for text in ["pci:01:08:02/1b36:0010", "pci:0c:03:30", "pci:ff:00:00/1234:11e8", "pci:00:00:00"] { + let parsed = PciMatch::parse(text).unwrap_or_else(|| panic!("{text} did not parse")); + assert_eq!(parsed.to_string(), text); + assert_eq!(Entry::parse(text), Some(Entry::Pci(parsed))); + } + assert_eq!(PciMatch::parse("pci:01:08:02/1b36:0010"), Some(NVME)); + } + + /// One spelling per matcher, because the label a claim is endowed under is + /// the entry's text and a driver looks it up by its own. + #[test] + fn a_matcher_has_one_spelling() { + for bad in [ + "pci:1b36:0010", // a card without its class + "pci:01:08", "pci:01:08:02:00", "pci:1:08:02", "pci:01:08:0A", "pci:01:08:02/", + "pci:01:08:02/1B36:0010", "pci:01:08:02/1b36", "pci:01:08:02/1b36:0010/x", "pci:+1:08:02", + "pci:", "pci", + ] { + assert_eq!(PciMatch::parse(bad), None, "{bad} parsed"); + assert_eq!(Entry::parse(bad), None, "{bad} parsed as an entry"); + } + } + + #[test] + fn a_fixed_entry_is_the_abis_spelling() { + assert_eq!(Entry::parse("keyboard"), Some(Entry::Fixed(DeviceRequest::Class(DeviceType::Keyboard)))); + assert!(matches!(Entry::parse("isa:0060,0064:1,12"), Some(Entry::Fixed(DeviceRequest::Isa(_))))); + assert_eq!(Entry::parse("gpu"), None); + } + + #[test] + fn a_matcher_matches_its_class_and_its_card() { + assert!(NVME.matches(&function([0x01, 0x08, 0x02], 0x1b36, 0x0010))); + assert!(!NVME.matches(&function([0x01, 0x08, 0x02], 0x144d, 0xa808))); + assert!(!NVME.matches(&function([0x01, 0x08, 0x01], 0x1b36, 0x0010))); + let any_nvme = PciMatch::new([0x01, 0x08, 0x02], None); + assert!(any_nvme.matches(&function([0x01, 0x08, 0x02], 0x144d, 0xa808))); + } + + /// Whether two entries can name one device, decided from their text: the + /// build refuses a manifest where they can. + #[test] + fn two_matchers_overlap_where_one_function_can_answer_both() { + let entry = |text| Entry::parse(text).unwrap(); + let overlapping = [ + ("pci:01:08:02/1b36:0010", "pci:01:08:02/1b36:0010"), + ("pci:01:08:02", "pci:01:08:02/1b36:0010"), + ("pci:01:08:02/1b36:0010", "pci:01:08:02"), + ("pci:0c:03:30", "pci:0c:03:30"), + ("framebuffer", "framebuffer"), + ("isa:0060,0064:1,12", "isa:0060,0064:1,12"), + ("part:0FC63DAF-8483-4772-8E79-3D69D8477DE4", "part:0FC63DAF-8483-4772-8E79-3D69D8477DE4"), + ]; + for (a, b) in overlapping { + assert!(entry(a).overlaps(&entry(b)), "{a} and {b}"); + } + let apart = [ + ("pci:01:08:02/1b36:0010", "pci:01:08:02/144d:a808"), + ("pci:01:08:02", "pci:01:08:01"), + ("pci:01:08:02/1b36:0010", "pci:01:08:01/1b36:0010"), + ("framebuffer", "keyboard"), + ("pci:0c:03:30", "keyboard"), + ]; + for (a, b) in apart { + assert!(!entry(a).overlaps(&entry(b)), "{a} and {b}"); + } + } +} diff --git a/lib/manifest/src/lib.rs b/lib/manifest/src/lib.rs index d31a73973a9..15ef6337701 100644 --- a/lib/manifest/src/lib.rs +++ b/lib/manifest/src/lib.rs @@ -18,7 +18,8 @@ //! serve the supervisor makes one machine-wide port and endows the acceptor //! provide this program makes its own port, once per instance //! receive a connector in this program's namespace -//! device a claim the supervisor mints and endows +//! device a claim the supervisor mints and endows ([`device`]) +//! drive one process of it per PCI function the matcher names //! syscap a right on the SysCap dup the supervisor endows //! slots the idle slot's partitions and the slot table //! service a system service: its `HOME` is `/state/`, not the session's @@ -32,7 +33,8 @@ //! login a launch it makes from the machine's session opens a login session //! ``` //! -//! [`package`] is the other half: what an installed package says about itself, +//! [`device`] is what a `devices` or `drives` entry names. [`package`] is the +//! other half: what an installed package says about itself, //! which is which of its own binaries a launch starts and never what it holds. //! [`launch`] is who may start what, [`grants`] which folder of the home a //! package is granted, [`consent`] how the person at the screen is asked, and @@ -40,6 +42,7 @@ pub mod clipboard; pub mod consent; +pub mod device; pub mod grants; pub mod launch; pub mod package; @@ -154,21 +157,6 @@ fn data_dir(dir: String, write: bool) -> View { View { role, dir, root, write } } -/// How often a `restart` row is started again before the supervisor gives up on it: at -/// most this many ends inside [`RESTART_WINDOW_SECS`]. Past it the row's ports -/// close, and a client's next connection is answered `Gone`. -pub const RESTARTS: u32 = 3; -pub const RESTART_WINDOW_SECS: u64 = 10; - -/// One start of a row, as the supervisor asks [`Program::supervised`] about it. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -pub enum Start { - /// It was just spawned. - Spawned, - /// It ended with this exit code. - Ended(i32), -} - pub use toyos_abi::handle::Rights; pub use toyos_abi::syscall::{DeviceRequest, DeviceType}; @@ -248,7 +236,13 @@ pub struct Program { pub provides: Vec, /// Names in this program's namespace, each a connector. pub receives: Vec, + /// Claims the supervisor mints for the one process of this row ([`device::Entry`]). pub devices: Vec, + /// PCI matchers ([`device::PciMatch`]): the supervisor starts one process + /// of this row per function one names and the kernel does not drive, + /// holding that function's claim. Such a row serves nothing and is never + /// in [`Manifest::start`]; it reaches its consumer through `receives`. + pub drives: Vec, /// Rights on the `SysCap` duplicate the supervisor endows this program, by the names /// [`syscap_rights`] takes. Empty for all but a handful: nothing else in /// the system may enter the RT band, mint a device claim, read the machine @@ -268,8 +262,8 @@ pub struct Program { /// the binary once per role, with the role as its argument and the /// acceptor of the role's port ([`role_dirs`] are the grants on it). pub roles: Vec, - /// The supervisor starts it again when it ends, on the same ports, for as long as - /// it does not end faster than [`RESTARTS`] allows. + /// The supervisor starts it again when it ends other than by exit 0, on the + /// same ports or function, within the supervisor's restart budget. pub restart: bool, /// The folder of the session's home a package's launch was granted /// ([`grants::decide`]): set by the supervisor on the row it synthesizes @@ -285,18 +279,6 @@ pub struct Program { } impl Program { - /// Whether the supervisor acts on `start` of this row. At its spawn, - /// whether a waiter sees it end: it serves ports, which close with an end - /// no restart is owed, or it is `restart`. At its end, whether it is - /// started again: it is `restart`, and it ended other than by exit 0, - /// which a program says only when it has nothing to serve. - pub fn supervised(&self, start: Start) -> bool { - match start { - Start::Spawned => self.restart || !self.serves.is_empty() || !self.roles.is_empty(), - Start::Ended(exit) => self.restart && exit != 0, - } - } - /// It replaces what the machine runs — the swap port's holder, or the idle /// slot's — so it starts only in a login session. pub fn login_only(&self) -> bool { @@ -415,9 +397,19 @@ pub enum RenderError { ServesWithoutService(String), /// A `roles` entry naming no file-server role. NoSuchRole { program: String, role: String }, + /// A `devices` entry that names no device, or a `drives` entry that is + /// no PCI matcher. + NoSuchDevice { program: String, entry: String }, + /// Two entries one device can answer, by row and entry: the second claim + /// would be refused at boot, or two rows would race for one function. + Overlap { first: (String, String), second: (String, String) }, + /// A `drives` row that also has what one process per function cannot + /// carry: `serves`, `roles` or `devices`, or a place in `[boot] start`. + Drives { program: String, also: &'static str }, } pub fn render(manifest: &Manifest) -> Result, RenderError> { + claims_apart(manifest)?; let mut out = String::new(); for program in &manifest.programs { if program.name.len() > MAX_PROGRAM_NAME { @@ -438,6 +430,7 @@ pub fn render(manifest: &Manifest) -> Result, RenderError> { ("provides", &program.provides), ("receives", &program.receives), ("devices", &program.devices), + ("drives", &program.drives), ("syscap", &program.syscap), ] { let word = match field { @@ -445,6 +438,7 @@ pub fn render(manifest: &Manifest) -> Result, RenderError> { "provides" => "provide", "receives" => "receive", "devices" => "device", + "drives" => "drive", _ => "syscap", }; for value in values { @@ -493,6 +487,42 @@ pub fn render(manifest: &Manifest) -> Result, RenderError> { Ok(out.into_bytes()) } +/// Every entry names a device, a `drives` row is one process per function and +/// nothing else, and no device can answer two entries. +fn claims_apart(manifest: &Manifest) -> Result<(), RenderError> { + let mut seen: Vec<(&str, &str, device::Entry)> = Vec::new(); + for program in &manifest.programs { + let refused = |entry: &String| RenderError::NoSuchDevice { program: program.name.clone(), entry: entry.clone() }; + if !program.drives.is_empty() { + let also = [ + (!program.serves.is_empty(), "serves"), + (!program.roles.is_empty(), "roles"), + (!program.devices.is_empty(), "devices"), + (manifest.start.contains(&program.name), "[boot] start"), + ]; + if let Some((_, also)) = also.into_iter().find(|(has, _)| *has) { + return Err(RenderError::Drives { program: program.name.clone(), also }); + } + } + let devices = program.devices.iter().map(|e| device::Entry::parse(e).ok_or_else(|| refused(e)).map(|p| (e, p))); + let drives = program.drives.iter().map(|e| match device::Entry::parse(e) { + Some(parsed @ device::Entry::Pci(_)) => Ok((e, parsed)), + _ => Err(refused(e)), + }); + for entry in devices.chain(drives) { + let (text, parsed) = entry?; + if let Some((row, other, _)) = seen.iter().find(|(_, _, earlier)| earlier.overlaps(&parsed)) { + return Err(RenderError::Overlap { + first: (row.to_string(), other.to_string()), + second: (program.name.clone(), text.clone()), + }); + } + seen.push((&program.name, text, parsed)); + } + } + Ok(()) +} + /// A field that becomes a whole record: no whitespace at all, because the /// parser splits the record word off at the first space. fn check(program: &str, field: &'static str, value: &str) -> Result<(), RenderError> { @@ -562,6 +592,7 @@ pub fn parse(text: &str) -> Manifest { "provide" => program.provides.push(rest.to_string()), "receive" => program.receives.push(rest.to_string()), "device" => program.devices.push(rest.to_string()), + "drive" => program.drives.push(rest.to_string()), "syscap" => program.syscap.push(rest.to_string()), "slots" if rest.is_empty() => program.slots = true, "service" => program.service = true, @@ -581,47 +612,6 @@ pub fn parse(text: &str) -> Manifest { mod tests { use super::*; - fn row(serves: &[&str], roles: &[&str], restart: bool) -> Program { - Program { - serves: serves.iter().map(|s| s.to_string()).collect(), - roles: roles.iter().map(|s| s.to_string()).collect(), - restart, - ..Program::default() - } - } - - /// A `restart` row that serves no port has no port to close, and is the - /// one row a waiter for ported starts alone would never see end. - #[test] - fn a_restart_row_with_no_port_is_waited_on() { - assert!(row(&[], &[], true).supervised(Start::Spawned)); - } - - #[test] - fn a_row_with_ports_is_waited_on_and_one_that_is_neither_is_not() { - assert!(row(&["block"], &[], false).supervised(Start::Spawned)); - assert!(row(&[], &["data"], false).supervised(Start::Spawned)); - assert!(!row(&[], &[], false).supervised(Start::Spawned)); - } - - #[test] - fn a_restart_rows_end_other_than_exit_0_is_owed_a_start_and_exit_0_is_not() { - for exit in [1, -1, 101, 139] { - assert!(row(&[], &[], true).supervised(Start::Ended(exit)), "exit {exit}"); - assert!(row(&["block"], &[], true).supervised(Start::Ended(exit)), "exit {exit}"); - } - assert!(!row(&[], &[], true).supervised(Start::Ended(0))); - assert!(!row(&["block"], &[], true).supervised(Start::Ended(0))); - } - - #[test] - fn a_row_that_is_not_restart_is_never_owed_a_start() { - for exit in [0, 1, 139] { - assert!(!row(&["log"], &[], false).supervised(Start::Ended(exit)), "exit {exit}"); - assert!(!row(&[], &[], false).supervised(Start::Ended(exit)), "exit {exit}"); - } - } - fn sample() -> Manifest { Manifest { programs: vec![ @@ -640,7 +630,7 @@ mod tests { name: "soundserver".into(), path: "/system/bin/soundserver".into(), serves: vec!["soundserver".into()], - devices: vec!["hda-audio".into(), "pci:1af4:1059".into()], + devices: vec!["hda-audio".into(), "pci:04:01:00/1af4:1059".into()], syscap: vec!["rt".into()], service: true, ..Program::default() @@ -659,6 +649,13 @@ mod tests { restart: true, ..Program::default() }, + Program { + name: "vdagent".into(), + path: "/system/bin/vdagent".into(), + receives: vec!["clipboard".into()], + drives: vec!["pci:07:80:00/1af4:1043".into()], + ..Program::default() + }, Program { name: "terminal".into(), path: "/system/bin/terminal".into(), @@ -919,6 +916,88 @@ mod tests { ); } + fn with_rows(rows: &[(&str, &[&str], &[&str])]) -> Manifest { + Manifest { + programs: rows + .iter() + .map(|(name, devices, drives)| Program { + name: name.to_string(), + path: format!("/system/bin/{name}"), + devices: devices.iter().map(|d| d.to_string()).collect(), + drives: drives.iter().map(|d| d.to_string()).collect(), + ..Program::default() + }) + .collect(), + ..Manifest::default() + } + } + + fn overlap(first: (&str, &str), second: (&str, &str)) -> Result, RenderError> { + Err(RenderError::Overlap { + first: (first.0.into(), first.1.into()), + second: (second.0.into(), second.1.into()), + }) + } + + /// **Two entries one device can answer are refused where the image is + /// built, naming both**: across rows and within one, a `devices` entry + /// and a `drives` matcher alike, and a fixed name written twice. + #[test] + fn two_entries_one_device_can_answer_are_refused_by_name() { + let cases: [(&[(&str, &[&str], &[&str])], _, _); 7] = [ + (&[("a", &["framebuffer"], &[]), ("b", &["framebuffer"], &[])], ("a", "framebuffer"), ("b", "framebuffer")), + (&[("a", &["isa:0060,0064:1,12"], &[]), ("b", &["isa:0060,0064:1,12"], &[])], ("a", "isa:0060,0064:1,12"), ("b", "isa:0060,0064:1,12")), + ( + &[("a", &["part:0FC63DAF-8483-4772-8E79-3D69D8477DE4"], &[]), ("b", &["part:0FC63DAF-8483-4772-8E79-3D69D8477DE4"], &[])], + ("a", "part:0FC63DAF-8483-4772-8E79-3D69D8477DE4"), + ("b", "part:0FC63DAF-8483-4772-8E79-3D69D8477DE4"), + ), + (&[("a", &["pci:01:08:02/1b36:0010"], &[]), ("b", &[], &["pci:01:08:02"])], ("a", "pci:01:08:02/1b36:0010"), ("b", "pci:01:08:02")), + (&[("a", &[], &["pci:0c:03:30"]), ("b", &[], &["pci:0c:03:30/1b36:000d"])], ("a", "pci:0c:03:30"), ("b", "pci:0c:03:30/1b36:000d")), + (&[("a", &["pci:02:00:00", "pci:02:00:00/1af4:1041"], &[])], ("a", "pci:02:00:00"), ("a", "pci:02:00:00/1af4:1041")), + (&[("a", &[], &["pci:ff:00:00", "pci:ff:00:00"])], ("a", "pci:ff:00:00"), ("a", "pci:ff:00:00")), + ]; + for (rows, first, second) in cases { + assert_eq!(render(&with_rows(rows)), overlap(first, second), "{rows:?}"); + } + let apart = with_rows(&[ + ("a", &["framebuffer", "pci:01:08:02/1b36:0010"], &[]), + ("b", &["keyboard", "pci:01:08:02/144d:a808"], &[]), + ("c", &[], &["pci:0c:03:30"]), + ]); + assert!(render(&apart).is_ok()); + } + + #[test] + fn an_entry_that_names_no_device_is_refused() { + for (devices, drives, entry) in [ + (&["gpu"][..], &[][..], "gpu"), + (&["pci:1af4:1041"], &[], "pci:1af4:1041"), + (&[], &["keyboard"], "keyboard"), + (&[], &["pci:01:08"], "pci:01:08"), + ] { + let refused = RenderError::NoSuchDevice { program: "a".into(), entry: entry.into() }; + assert_eq!(render(&with_rows(&[("a", devices, drives)])), Err(refused)); + } + } + + /// A `drives` row is one process per function, so it serves no port, holds + /// no claim but its function's and is started by no `[boot] start`. + #[test] + fn a_drives_row_is_one_process_per_function_and_nothing_else() { + let drives = |change: fn(&mut Manifest)| { + let mut m = with_rows(&[("a", &[], &["pci:ff:00:00/1234:11e8"])]); + change(&mut m); + render(&m) + }; + let refused = |also| Err(RenderError::Drives { program: "a".into(), also }); + assert!(drives(|_| {}).is_ok()); + assert_eq!(drives(|m| m.programs[0].serves = vec!["edu".into()]), refused("serves")); + assert_eq!(drives(|m| m.programs[0].roles = vec!["data".into()]), refused("roles")); + assert_eq!(drives(|m| m.programs[0].devices = vec!["keyboard".into()]), refused("devices")); + assert_eq!(drives(|m| m.start = vec!["a".into()]), refused("[boot] start")); + } + #[test] fn a_device_class_name_is_the_abi_s() { assert_eq!(DeviceType::from_class_name("hda-audio"), Some(DeviceType::HdaAudio)); diff --git a/sdk/abi/src/inventory.rs b/sdk/abi/src/inventory.rs index d819aa2410c..2fa794ad570 100644 --- a/sdk/abi/src/inventory.rs +++ b/sdk/abi/src/inventory.rs @@ -61,6 +61,11 @@ pub struct PciAddr { pub func: u8, } +/// A PCI function's number for this boot, never given to another: what a +/// claim names it by ([`crate::syscall::DeviceRequest::Function`]). +#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug)] +pub struct Arrival(pub u32); + /// Who drives a PCI function. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum Driven { @@ -76,6 +81,7 @@ pub enum Driven { /// One PCI function. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub struct Pci { + pub arrival: Arrival, pub at: PciAddr, pub vendor: u16, pub device: u16, @@ -344,6 +350,7 @@ impl Record { Driven::Claimed => 2, }, ); + o.u32(16, p.arrival.0); } Self::Usb(u) => { o.u8(0, KIND_USB); @@ -422,6 +429,7 @@ impl Record { let r = In(&raw.0); Ok(match r.u8(0) { KIND_PCI => Self::Pci(Pci { + arrival: Arrival(r.u32(16)), at: r.addr(1), vendor: r.u16(6), device: r.u16(8), @@ -506,6 +514,7 @@ mod tests { let holder = Holder { pid: 7, name: name("netstack") }; [ Record::Pci(Pci { + arrival: Arrival(0x0102_0304), at, vendor: 0x8086, device: 0x15fc, diff --git a/sdk/abi/src/syscall.rs b/sdk/abi/src/syscall.rs index 0514d57bd49..f98cf28fd3e 100644 --- a/sdk/abi/src/syscall.rs +++ b/sdk/abi/src/syscall.rs @@ -1330,9 +1330,10 @@ device_classes! { /// driver to it, keeps config space, puts it in an address space of its own /// before it lets it master the bus, and programs its interrupt vector. /// - /// The one class whose name is not the whole of what a config writes — - /// `pci::` names which function, and [`DeviceRequest`] is - /// the one parser of that spelling. + /// A claim names which function by its + /// [`Arrival`](crate::inventory::Arrival), and no name is spelled for it + /// here: which functions a manifest row asks for is `toyos-manifest`'s + /// grammar, resolved against the inventory. PciFunction = 7 => "pci", /// One GPT partition, named by its unique partition GUID: the holder reads /// and writes its blocks and no other's ([`crate::part`]). @@ -1354,11 +1355,10 @@ device_classes! { Acpi = 10 => "acpi", } -/// A PCI function named by what identifies the *card*, not the slot firmware -/// put it in: the same card is `00:03.0` on one machine and `00:1f.6` on -/// another, so a name that was a position would hand a program whatever was -/// there. A machine holding two of one card has an ambiguous name, which -/// [`device_claim`] refuses rather than resolving. +/// What identifies a PCI *card*, not the slot firmware put it in: the same +/// card is `00:03.0` on one machine and `00:1f.6` on another. Two of one card +/// carry one id, so it names no single function, and a claim names one by its +/// [`Arrival`](crate::inventory::Arrival). #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub struct PciId { pub vendor: u16, @@ -1373,12 +1373,12 @@ impl PciId { Some(Self { vendor: hex16(vendor)?, device: hex16(device)? }) } - /// The selector word [`device_claim`] carries. + /// The id as one word. pub const fn wire(self) -> u64 { ((self.vendor as u64) << 16) | self.device as u64 } - /// The selector word decoded; `None` for anything with bits above 32. + /// [`Self::wire`] decoded; `None` for anything with bits above 32. pub const fn from_wire(raw: u64) -> Option { if raw > u32::MAX as u64 { return None; @@ -1389,9 +1389,9 @@ impl PciId { /// Exactly four lowercase hex digits and nothing else. /// -/// One spelling per function, because every reader of a `devices` entry -/// compares the string: `1af4:41` and `1af4:0041`, or `1af4:1041` and -/// `1AF4:1041`, would otherwise be two names for one card. +/// One spelling per value, because every reader of a name compares the +/// string: `1af4:41` and `1af4:0041`, or `1af4:1041` and `1AF4:1041`, would +/// otherwise be two names for one card. fn hex16(text: &str) -> Option { if text.len() != 4 { return None; @@ -1533,26 +1533,24 @@ fn decimal_line(text: &str) -> Option { (value < IsaId::LINES).then_some(value) } -/// What one `devices` entry asks for: a class, and for -/// [`DeviceType::PciFunction`] which function, for a partition which -/// partition, and for an ISA function which ports and lines. +/// What a claim asks for: a class, and for [`DeviceType::PciFunction`] which +/// function, for a partition which partition, and for an ISA function which +/// ports and lines. /// -/// One parser, because four places read the same spelling — the build system's -/// gate, `/system/bin/supervisor`'s mint, the kernel's claim and the claimant's own +/// Every request but a PCI function's is also a `devices` entry, spelled once +/// here, because four places read that spelling — the build system's gate, +/// `/system/bin/supervisor`'s mint, the kernel's claim and the claimant's own /// lookup — and a second would be a name a config can write and a program /// cannot find. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum DeviceRequest { Class(DeviceType), - Pci(PciId), + Function(crate::inventory::Arrival), Partition(crate::part::PartGuid), Isa(IsaId), } impl DeviceRequest { - /// The spelling that carries a function id after the class name. - pub const PCI_PREFIX: &'static str = "pci:"; - /// The spelling that carries a unique partition GUID. pub const PART_PREFIX: &'static str = "part:"; @@ -1563,10 +1561,9 @@ impl DeviceRequest { /// `part:` and a GUID, rounded up to a multiple of eight. pub const MAX_NAME: usize = 48; + /// The request a `devices` entry spells, or `None`; never a + /// [`Self::Function`], which has no spelling. pub fn parse(name: &str) -> Option { - if let Some(id) = name.strip_prefix(Self::PCI_PREFIX) { - return PciId::parse(id).map(Self::Pci); - } if let Some(guid) = name.strip_prefix(Self::PART_PREFIX) { return crate::part::PartGuid::parse(guid).map(Self::Partition); } @@ -1585,7 +1582,7 @@ impl DeviceRequest { pub const fn class(self) -> DeviceType { match self { Self::Class(class) => class, - Self::Pci(_) => DeviceType::PciFunction, + Self::Function(_) => DeviceType::PciFunction, Self::Partition(_) => DeviceType::Partition, Self::Isa(_) => DeviceType::Isa, } @@ -1596,22 +1593,24 @@ impl DeviceRequest { pub fn selector(self) -> [u64; 2] { match self { Self::Class(_) => [0, 0], - Self::Pci(id) => [id.wire(), 0], + Self::Function(arrival) => [arrival.0 as u64, 0], Self::Partition(guid) => guid.wire(), Self::Isa(set) => set.wire(), } } /// This request's `devices` spelling written into `buf`, which - /// [`Self::MAX_NAME`] bounds. The name is also the tail of the `dev:` label - /// the claim arrives under, and it is composed here so the two cannot drift. - pub fn write_name(self, buf: &mut [u8; Self::MAX_NAME]) -> &str { + /// [`Self::MAX_NAME`] bounds, or `None` for a [`Self::Function`], which has + /// none. The name is also the tail of the `dev:` label the claim arrives + /// under, and it is composed here so the two cannot drift. + pub fn write_name(self, buf: &mut [u8; Self::MAX_NAME]) -> Option<&str> { use crate::part::GUID_TEXT_LEN; - let id = match self { + Some(match self { + Self::Function(_) => return None, Self::Class(class) => { let name = class.class_name(); buf[..name.len()].copy_from_slice(name.as_bytes()); - return core::str::from_utf8(&buf[..name.len()]).expect("a class name is ASCII"); + core::str::from_utf8(&buf[..name.len()]).expect("a class name is ASCII") } Self::Partition(guid) => { let at = Self::PART_PREFIX.len(); @@ -1619,8 +1618,7 @@ impl DeviceRequest { let mut text = [0u8; GUID_TEXT_LEN]; guid.write_text(&mut text); buf[at..at + GUID_TEXT_LEN].copy_from_slice(&text); - return core::str::from_utf8(&buf[..at + GUID_TEXT_LEN]) - .expect("a prefix and a GUID's text are ASCII"); + core::str::from_utf8(&buf[..at + GUID_TEXT_LEN]).expect("a prefix and a GUID's text are ASCII") } Self::Isa(set) => { let mut at = Self::ISA_PREFIX.len(); @@ -1649,24 +1647,9 @@ impl DeviceRequest { buf[at] = b'0' + line % 10; at += 1; } - return core::str::from_utf8(&buf[..at]).expect("hex, digits and separators are ASCII"); + core::str::from_utf8(&buf[..at]).expect("hex, digits and separators are ASCII") } - Self::Pci(id) => id, - }; - let prefix = Self::PCI_PREFIX.as_bytes(); - buf[..prefix.len()].copy_from_slice(prefix); - let mut at = prefix.len(); - for value in [id.vendor, id.device] { - if at != prefix.len() { - buf[at] = b':'; - at += 1; - } - for shift in [12, 8, 4, 0] { - buf[at] = HEX[((value >> shift) & 0xF) as usize]; - at += 1; - } - } - core::str::from_utf8(&buf[..at]).expect("hex digits and colons are ASCII") + }) } } @@ -1674,10 +1657,10 @@ const HEX: &[u8; 16] = b"0123456789abcdef"; /// Mint a device claim for `request`, presenting a `SysCap` handle that carries /// [`Rights::DEVICE`]. `NotFound` for a class no driver registered, or a PCI -/// function or partition this machine does not have — the supervisor endows what exists +/// arrival or partition this machine does not have — the supervisor endows what exists /// and logs what it did not. `AlreadyExists` names one already claimed, and /// `PermissionDenied` one the kernel drives itself — for a partition, one it -/// has mounted. A request matching more than one function or partition answers +/// has mounted. A partition GUID two partitions carry answers /// `InvalidArgument`, because an ambiguous name is refused rather than resolved /// to the first match; so does a nonzero selector word the class does not read. /// @@ -2776,7 +2759,7 @@ mod tests { } /// Four lowercase hex digits each and nothing else, because two spellings - /// are two claims on one function: `1af4:41` beside `1af4:0041`, and + /// would be two names for one card: `1af4:41` beside `1af4:0041`, and /// `1AF4:1041` beside `1af4:1041`. #[test] fn a_pci_id_is_four_lowercase_hex_digits_each_and_nothing_else() { @@ -2791,13 +2774,14 @@ mod tests { } /// The four readers of a `devices` entry share this parser, so what it - /// accepts is what a config may say. + /// accepts is what a config may say; a PCI function is named by its + /// arrival, which no entry spells. #[test] - fn a_devices_entry_is_a_class_or_one_named_function() { - let want = DeviceRequest::Pci(PciId { vendor: 0x1af4, device: 0x1041 }); - assert_eq!(DeviceRequest::parse("pci:1af4:1041"), Some(want)); - assert_eq!(want.class(), DeviceType::PciFunction); - assert_eq!(want.selector(), [0x1af4_1041, 0]); + fn a_devices_entry_is_a_class_or_one_named_device() { + let function = DeviceRequest::Function(crate::inventory::Arrival(7)); + assert_eq!(function.class(), DeviceType::PciFunction); + assert_eq!(function.selector(), [7, 0]); + assert_eq!(function.write_name(&mut [0; DeviceRequest::MAX_NAME]), None); assert_eq!( DeviceRequest::parse("framebuffer"), Some(DeviceRequest::Class(DeviceType::Framebuffer)) @@ -2806,9 +2790,8 @@ mod tests { // Every one of these is a name a config could plausibly write and the // kernel must not resolve to a function. for bad in [ - "pci", // the bare class name names no function at all - "pci:", // nor does the prefix on its own - "pci:1af4", + "pci", // the bare class name names no function at all + "pci:1af4:1041", // and a function's matcher is the manifest's "gpu", // a class name this table has never had "part", // a bare partition class names no partition "part:", @@ -2842,9 +2825,6 @@ mod tests { #[test] fn a_request_writes_back_the_name_it_parsed() { for name in [ - "pci:1af4:1041", - "pci:8086:15fc", - "pci:0000:0000", "hda-audio", "mouse", // The longest entry there is, which is what `MAX_NAME` is for. @@ -2856,7 +2836,7 @@ mod tests { ] { let request = DeviceRequest::parse(name).expect("a name this table has"); let mut buf = [0u8; DeviceRequest::MAX_NAME]; - assert_eq!(request.write_name(&mut buf), name); + assert_eq!(request.write_name(&mut buf), Some(name)); } } @@ -2897,10 +2877,10 @@ mod tests { } } - /// The selector is one word on the wire and the kernel decodes it back; - /// a vendor lost to a shift would claim a different card. + /// The id is one word and decodes back; a vendor lost to a shift would + /// name a different card. #[test] - fn the_selector_survives_the_wire() { + fn a_pci_id_survives_its_word() { for id in [ PciId { vendor: 0x1af4, device: 0x1041 }, PciId { vendor: 0x8086, device: 0x15fc }, diff --git a/sdk/toyos/src/endow.rs b/sdk/toyos/src/endow.rs index 51b23edd1e5..37df000e3d7 100644 --- a/sdk/toyos/src/endow.rs +++ b/sdk/toyos/src/endow.rs @@ -228,31 +228,27 @@ pub fn device(class: DeviceType) -> Option { with_prefixed(DEV_PREFIX, class.class_name(), |label| Endowments::get().take::(label)) } -/// The claim for one PCI function the manifest says this program gets. -/// -/// The label is the `devices` entry's own spelling, written by the one -/// [`DeviceRequest`] that also parsed it: a program looking up `pci:1af4:1041` -/// and a config declaring it cannot disagree about what the name is. -pub fn pci_function(id: toyos_abi::syscall::PciId) -> Option { - let mut buf = [0u8; DeviceRequest::MAX_NAME]; - let name = DeviceRequest::Pci(id).write_name(&mut buf); +/// The claim the manifest's entry `name` says this program gets, under the +/// label the supervisor endows it by: the entry's own spelling. A PCI +/// function's is `toyos-manifest`'s matcher, which a driver spells with the +/// same type the manifest is parsed with. +pub fn claim(name: &str) -> Option { with_prefixed(DEV_PREFIX, name, |label| Endowments::get().take::(label)) } /// The claim for one partition the manifest says this program gets, looked up -/// by the entry's own spelling exactly as [`pci_function`] is. +/// by the entry's own spelling, written by the one [`DeviceRequest`] that also +/// parsed it. pub fn partition(guid: toyos_abi::part::PartGuid) -> Option { let mut buf = [0u8; DeviceRequest::MAX_NAME]; - let name = DeviceRequest::Partition(guid).write_name(&mut buf); - with_prefixed(DEV_PREFIX, name, |label| Endowments::get().take::(label)) + claim(DeviceRequest::Partition(guid).write_name(&mut buf)?) } /// The claim for one legacy ISA function the manifest says this program gets, -/// looked up by the entry's own spelling exactly as [`pci_function`] is. +/// looked up by the entry's own spelling exactly as [`partition`] is. pub fn isa(set: toyos_abi::syscall::IsaId) -> Option { let mut buf = [0u8; DeviceRequest::MAX_NAME]; - let name = DeviceRequest::Isa(set).write_name(&mut buf); - with_prefixed(DEV_PREFIX, name, |label| Endowments::get().take::(label)) + claim(DeviceRequest::Isa(set).write_name(&mut buf)?) } /// Compose `` on the stack. Nothing in the SDK allocates, and a diff --git a/sdk/toyos/src/syscap.rs b/sdk/toyos/src/syscap.rs index a90cda0dc37..526485b712a 100644 --- a/sdk/toyos/src/syscap.rs +++ b/sdk/toyos/src/syscap.rs @@ -26,23 +26,24 @@ impl SysCap { self.mint(DeviceRequest::Class(class)) } - /// Mint the claim for one PCI function, named by what identifies the card. + /// Mint the claim for one PCI function, by the arrival its inventory + /// record carries. /// /// Apart from [`Self::claim`] rather than one call taking a /// [`DeviceRequest`], so there is no way to ask for /// [`DeviceType::PciFunction`] with no function named: a class is the whole - /// of what the call above asks for, and this one cannot be asked without an - /// id. - pub fn claim_pci( + /// of what the call above asks for, and this one cannot be asked without + /// an arrival. + pub fn claim_function( &self, - id: toyos_abi::syscall::PciId, + arrival: toyos_abi::inventory::Arrival, ) -> Result { - self.mint(DeviceRequest::Pci(id)) + self.mint(DeviceRequest::Function(arrival)) } /// Mint the claim for one GPT partition, by its unique GUID. Apart from - /// [`Self::claim`] for the reason [`Self::claim_pci`] is: this one cannot be - /// asked without naming which. + /// [`Self::claim`] for the reason [`Self::claim_function`] is: this one + /// cannot be asked without naming which. /// /// `PermissionDenied` is a partition the kernel has mounted, and /// `AlreadyExists` one another process holds: a partition has one holder. @@ -54,7 +55,7 @@ impl SysCap { } /// Mint the claim for one legacy ISA function, by exactly its ports and - /// lines. Apart from [`Self::claim`] for the reason [`Self::claim_pci`] is. + /// lines. Apart from [`Self::claim`] for the reason [`Self::claim_function`] is. /// /// `NotFound` is a set that is not one function this machine can hand out /// whole, and `PermissionDenied` one the kernel drives itself. diff --git a/system/netstack/Cargo.toml b/system/netstack/Cargo.toml index 73dad107252..6fb3a3a3b0a 100644 --- a/system/netstack/Cargo.toml +++ b/system/netstack/Cargo.toml @@ -6,6 +6,8 @@ license = "MIT OR Apache-2.0" [dependencies] toyos-abi = { path = "../../sdk/abi" } +# The matcher its claim is endowed under, spelled as the manifest spells it. +toyos-manifest = { path = "../../lib/manifest" } toyos = { path = "../../sdk/toyos" } toyos-device-memory = { path = "../../lib/device-memory" } toyos-i219 = { path = "i219" } diff --git a/system/netstack/src/main.rs b/system/netstack/src/main.rs index 05bbf0172a8..70ddb3a33c5 100644 --- a/system/netstack/src/main.rs +++ b/system/netstack/src/main.rs @@ -33,6 +33,7 @@ use toyos::poller::{Poller, READABLE}; use toyos::say; use toyos::AsHandle; use toyos_abi::syscall::PciId; +use toyos_manifest::device::PciMatch; use toyos_dhcp::{HostName, Lease}; use toyos_i219::Part; use toyos_inspect::Snapshot; @@ -53,24 +54,29 @@ mod watches; use card::Card; use client::{Client, ClientRx, HANDSHAKE_TIMEOUT, MAX_KEPT_REQUEST, MAX_PENDING_CONNS, PendingConn, Request}; -/// The cards this program can drive, named by what identifies one rather than -/// by the slot firmware put it in, and each with the driver that opens it. The -/// manifest row spells the same pair and the claim arrives under a label -/// composed from it, so which of these exists is `/system/bin/supervisor`'s answer -/// and not this program's — at most one is ever endowed, and a machine with -/// none is a machine netstack leaves. +/// The cards this program can drive, matched by their class and what +/// identifies the card rather than by the slot firmware put it in, and each +/// with the driver that opens it. The manifest row spells the same matcher and +/// the claim arrives under its text, so which of these exists is +/// `/system/bin/supervisor`'s answer and not this program's — at most one is +/// ever endowed, and a machine with none is a machine netstack leaves. /// /// `1af4:1041` is virtio's transitional device id `1000 + 1` for a network /// device (virtio 1.2 §5.1.1). `8086:15fc` is the ThinkPad T14's onboard I219 /// at `00:1f.6`; `8086:10d3` is the 82574L, which QEMU's `e1000e` models. One /// driver takes both, and each row names which part it is because below the /// register file they are not one. -const CARDS: [(PciId, fn(toyos::PciDev) -> Card); 3] = [ - (PciId { vendor: 0x8086, device: 0x15fc }, |c| Card::intel(c, Part::I219)), - (PciId { vendor: 0x8086, device: 0x10d3 }, |c| Card::intel(c, Part::E82574)), - (PciId { vendor: 0x1af4, device: 0x1041 }, Card::virtio), +const CARDS: [(PciMatch, fn(toyos::PciDev) -> Card); 3] = [ + (ethernet(0x8086, 0x15fc), |c| Card::intel(c, Part::I219)), + (ethernet(0x8086, 0x10d3), |c| Card::intel(c, Part::E82574)), + (ethernet(0x1af4, 0x1041), Card::virtio), ]; +/// An Ethernet controller, class `02:00:00`. +const fn ethernet(vendor: u16, device: u16) -> PciMatch { + PciMatch::new([0x02, 0x00, 0x00], Some(PciId { vendor, device })) +} + /// The name this machine asks its network to record for it, and answers to on /// it as `.local` once no other host does. One name, because there is /// one machine. @@ -307,7 +313,7 @@ fn main() { // than silence. let Some((open, claim)) = CARDS .iter() - .find_map(|(id, open)| endow::pci_function::(*id).map(|c| (*open, c))) + .find_map(|(card, open)| endow::claim::(&card.to_string()).map(|c| (*open, c))) else { say!("netstack: no NIC on this machine, exiting"); return; diff --git a/system/soundserver/Cargo.toml b/system/soundserver/Cargo.toml index bf3aa9887f2..8c83385fc10 100644 --- a/system/soundserver/Cargo.toml +++ b/system/soundserver/Cargo.toml @@ -6,6 +6,8 @@ license = "MIT OR Apache-2.0" [dependencies] toyos-abi = { path = "../../sdk/abi" } +# The matcher its claim is endowed under, spelled as the manifest spells it. +toyos-manifest = { path = "../../lib/manifest" } toyos = { path = "../../sdk/toyos" } toyos-hda = { path = "../../lib/hda" } toyos-mixer = { path = "mixer" } diff --git a/system/soundserver/src/main.rs b/system/soundserver/src/main.rs index e098079e776..20cf7747f28 100644 --- a/system/soundserver/src/main.rs +++ b/system/soundserver/src/main.rs @@ -87,7 +87,7 @@ fn main() { // // The order is virtio first, and it is not a preference between two cards: // no machine in this project has both. The T14 has only the second. - if let Some(dev) = endow::pci_function::(virtio::PCI_ID) { + if let Some(dev) = endow::claim::(&virtio::FUNCTION.to_string()) { match virtio::Virtio::claim(dev) { Ok((virtio, rate, channels)) => return run_virtio(ports, virtio, rate, channels), Err(why) => { diff --git a/system/soundserver/src/virtio.rs b/system/soundserver/src/virtio.rs index 1bd40d6def2..98c4c050422 100644 --- a/system/soundserver/src/virtio.rs +++ b/system/soundserver/src/virtio.rs @@ -30,9 +30,13 @@ use toyos_virtio_sound::{Queues, Sound, PERIOD_BYTES, STREAM_ID}; use crate::backend::Completion; -/// virtio's vendor id and the sound device's (§4.1.2.1: `0x1040` + 25), as -/// the manifest row spells the claim. -pub const PCI_ID: toyos_abi::syscall::PciId = toyos_abi::syscall::PciId { vendor: 0x1af4, device: 0x1059 }; +/// A multimedia audio controller, class `04:01:00`, of virtio's vendor id and +/// the sound device's (§4.1.2.1: `0x1040` + 25), as the manifest row spells +/// the claim. +pub const FUNCTION: toyos_manifest::device::PciMatch = toyos_manifest::device::PciMatch::new( + [0x04, 0x01, 0x00], + Some(toyos_abi::syscall::PciId { vendor: 0x1af4, device: 0x1059 }), +); /// The one MSI-X table entry the kernel programs. const MSIX_ENTRY: u16 = 0; diff --git a/system/supervisor/Cargo.toml b/system/supervisor/Cargo.toml index 349f3f2711c..b0cb8ddc01a 100644 --- a/system/supervisor/Cargo.toml +++ b/system/supervisor/Cargo.toml @@ -9,6 +9,9 @@ license = "MIT OR Apache-2.0" toyos = { path = "../../sdk/toyos" } toyos-abi = { path = "../../sdk/abi" } toyos-manifest = { path = "../../lib/manifest" } +# Its lifecycle decisions: what an end owes, which claims a start mints, and +# which functions a `drives` row has a process on. +toyos-supervisor = { path = "lifecycle" } toyos-osrelease = { path = "../../sdk/osrelease" } toyos-wallclock = { path = "../../lib/wallclock" } toyos-logstream = { path = "../../lib/logstream" } diff --git a/system/supervisor/lifecycle/Cargo.toml b/system/supervisor/lifecycle/Cargo.toml new file mode 100644 index 00000000000..7865a020c92 --- /dev/null +++ b/system/supervisor/lifecycle/Cargo.toml @@ -0,0 +1,16 @@ +[package] +name = "toyos-supervisor" +description = "The supervisor's lifecycle decisions, pure: which PCI functions a drives row has a process on, the function a devices entry names, and when an ended process is started again." +version = "0.1.0" +edition = "2021" +license = "MIT OR Apache-2.0" + +[lib] +doctest = false + +[dependencies] +toyos-abi = { path = "../../../sdk/abi" } +toyos-manifest = { path = "../../../lib/manifest" } + +[lints.rust] +warnings = "deny" diff --git a/system/supervisor/lifecycle/src/claims.rs b/system/supervisor/lifecycle/src/claims.rs new file mode 100644 index 00000000000..47dc08b86e1 --- /dev/null +++ b/system/supervisor/lifecycle/src/claims.rs @@ -0,0 +1,108 @@ +//! Which claims a start of a row with one process mints. +//! +//! A `devices` PCI matcher names the one function it matches that the kernel +//! does not drive, and the claim names that function by its arrival. **A start +//! that follows an end is owed every claim the process before it held**: one +//! it cannot mint is no start, so a service never runs on without a device it +//! had. + +use toyos_abi::inventory::{Arrival, Driven, Pci, PciAddr}; +use toyos_abi::syscall::SyscallError; +use toyos_manifest::device::PciMatch; + +/// Why a matcher names no one function. +#[derive(Clone, PartialEq, Eq, Debug)] +pub enum Unresolved { + /// No function matches it. + Absent, + /// Every function it matches is the kernel's. + KernelDriven(Vec), + /// It matches every one of these, and a row with one process cannot say + /// which. + Ambiguous(Vec), +} + +/// The function `matcher` names on a machine whose inventory is `records`. +pub fn function(matcher: &PciMatch, records: &[Pci]) -> Result { + let matched: Vec<&Pci> = records.iter().filter(|pci| matcher.matches(pci)).collect(); + let undriven: Vec<&Pci> = matched.iter().copied().filter(|pci| pci.driven != Driven::Kernel).collect(); + match (&undriven[..], &matched[..]) { + ([one], _) => Ok(one.arrival), + ([], []) => Err(Unresolved::Absent), + ([], driven) => Err(Unresolved::KernelDriven(driven.iter().map(|pci| pci.at).collect())), + (several, _) => Err(Unresolved::Ambiguous(several.iter().map(|pci| pci.at).collect())), + } +} + +/// Why a start holds no claim for an entry. +#[derive(Clone, PartialEq, Eq, Debug)] +pub enum Missing { + Unresolved(Unresolved), + /// The kernel refused the claim. + Refused(SyscallError), +} + +/// Whether a start goes on without `entry`'s claim. A matcher naming several +/// functions refuses the start, and so does a claim the process before it +/// held, `owed`. +pub fn goes_on_without(entry: &str, missing: &Missing, owed: &[String]) -> bool { + !matches!(missing, Missing::Unresolved(Unresolved::Ambiguous(_))) && !owed.iter().any(|held| held == entry) +} + +#[cfg(test)] +mod tests { + use super::*; + use toyos_abi::syscall::PciId; + + const EDU: PciMatch = PciMatch::new([0xff, 0x00, 0x00], Some(PciId { vendor: 0x1234, device: 0x11e8 })); + + fn edu(arrival: u32, dev: u8, driven: Driven) -> Pci { + Pci { + arrival: Arrival(arrival), + at: PciAddr { segment: 0, bus: 0, dev, func: 0 }, + vendor: 0x1234, + device: 0x11e8, + class: 0xff, + subclass: 0, + prog_if: 0, + driven, + } + } + + fn at(dev: u8) -> PciAddr { + PciAddr { segment: 0, bus: 0, dev, func: 0 } + } + + #[test] + fn a_matcher_names_the_one_function_the_kernel_does_not_drive() { + assert_eq!(function(&EDU, &[edu(3, 3, Driven::Kernel), edu(4, 4, Driven::Free)]), Ok(Arrival(4))); + // One another process holds is still the function named, and its + // claim is refused by the kernel, which says so. + assert_eq!(function(&EDU, &[edu(4, 4, Driven::Claimed)]), Ok(Arrival(4))); + assert_eq!(function(&EDU, &[]), Err(Unresolved::Absent)); + assert_eq!(function(&EDU, &[edu(3, 3, Driven::Kernel)]), Err(Unresolved::KernelDriven(vec![at(3)]))); + } + + /// Two functions a single-process row's matcher names are both named back, + /// and its start is refused whatever it was owed. + #[test] + fn two_matching_functions_refuse_the_start_naming_both() { + let two = function(&EDU, &[edu(3, 3, Driven::Free), edu(4, 4, Driven::Claimed)]); + assert_eq!(two, Err(Unresolved::Ambiguous(vec![at(3), at(4)]))); + let missing = Missing::Unresolved(two.unwrap_err()); + assert!(!goes_on_without("pci:ff:00:00/1234:11e8", &missing, &[])); + } + + /// A start that follows an end is refused every claim the process before + /// it held, and goes on without one it never had. + #[test] + fn a_start_goes_on_without_only_what_it_was_not_owed() { + let entry = "pci:02:00:00/1af4:1041"; + let owed = [entry.to_string()]; + for missing in [Missing::Unresolved(Unresolved::Absent), Missing::Refused(SyscallError::AlreadyExists)] { + assert!(goes_on_without(entry, &missing, &[]), "{missing:?}"); + assert!(!goes_on_without(entry, &missing, &owed), "{missing:?}"); + assert!(goes_on_without("keyboard", &missing, &owed), "{missing:?}"); + } + } +} diff --git a/system/supervisor/lifecycle/src/drives.rs b/system/supervisor/lifecycle/src/drives.rs new file mode 100644 index 00000000000..e1875310f24 --- /dev/null +++ b/system/supervisor/lifecycle/src/drives.rs @@ -0,0 +1,246 @@ +//! Which PCI functions a `drives` row has a process on. +//! +//! **One process per function a `drives` matcher names and the kernel does not +//! drive**, keyed by the function's arrival. The supervisor reads the inventory +//! at boot and after every end of a process it started, hands each reading to +//! [`Drives::decide`], and performs what it answers. +//! +//! **A reading that changed nothing starts nothing.** A start that was refused +//! — the claim, the hand-over or the spawn — and an end owed no restart leave +//! the function alone until its inventory record differs from the first one +//! read after. Every start of a function after its first spends its +//! [`Budget`], and a function whose budget is spent is given up on. + +use std::collections::BTreeMap; +use std::time::Instant; + +use toyos_abi::inventory::{Arrival, Driven, Pci}; +use toyos_manifest::device::PciMatch; +use toyos_manifest::{Manifest, Program}; + +use crate::restart::Budget; + +/// Every `drives` matcher in `manifest`, with its row. +/// +/// Panics on one that is no matcher: the build refuses to render such a +/// manifest, so it is an image whose halves disagree. +pub fn matchers(manifest: &Manifest) -> Vec<(&Program, PciMatch)> { + let parse = |row: &Program, entry: &String| { + PciMatch::parse(entry).unwrap_or_else(|| panic!("supervisor: {} drives `{entry}`, which is no matcher", row.name)) + }; + manifest.programs.iter().flat_map(|row| row.drives.iter().map(move |entry| (row, parse(row, entry)))).collect() +} + +/// What to do about one function. +#[derive(Debug, PartialEq, Eq)] +pub enum Act<'m> { + /// Start a process of `row` holding `function`'s claim, endowed under + /// `entry`'s spelling. + Start { row: &'m Program, entry: PciMatch, function: Pci }, + /// `function`'s budget is spent: it gets no process again. + GiveUp { row: &'m Program, function: Pci }, +} + +#[derive(Default)] +pub struct Drives { + functions: BTreeMap, +} + +#[derive(Default)] +struct Function { + budget: Budget, + started: bool, + phase: Phase, +} + +#[derive(Default)] +enum Phase { + /// A process is owed whenever the function is free. + #[default] + Owed, + /// A process holds it, or is being started on it. + Running, + /// Left alone until its record differs from this one: the first read after. + Held(Option), + GivenUp, +} + +impl Drives { + /// What to do about the functions `records` lists, which `rows` match. + pub fn decide<'m>(&mut self, rows: &[(&'m Program, PciMatch)], records: &[Pci], now: Instant) -> Vec> { + self.functions.retain(|arrival, function| { + matches!(function.phase, Phase::Running) || records.iter().any(|pci| pci.arrival == *arrival) + }); + let mut acts = Vec::new(); + for pci in records.iter().filter(|pci| pci.driven != Driven::Kernel) { + let Some(&(row, entry)) = rows.iter().find(|(_, matcher)| matcher.matches(pci)) else { continue }; + let function = self.functions.entry(pci.arrival).or_default(); + if let Phase::Held(seen) = &mut function.phase { + match seen { + None => *seen = Some(*pci), + Some(seen) if seen != pci => function.phase = Phase::Owed, + Some(_) => {} + } + } + if !matches!(function.phase, Phase::Owed) || pci.driven != Driven::Free { + continue; + } + if function.started && !function.budget.spend(now) { + function.phase = Phase::GivenUp; + acts.push(Act::GiveUp { row, function: *pci }); + continue; + } + function.started = true; + function.phase = Phase::Running; + acts.push(Act::Start { row, entry, function: *pci }); + } + acts + } + + /// The start [`Self::decide`] asked for on `arrival` did not happen. + pub fn refused(&mut self, arrival: Arrival) { + self.leave(arrival); + } + + /// The process on `arrival` ended; `owed` is whether its row owes it + /// another ([`crate::restart::owed`]). + pub fn ended(&mut self, arrival: Arrival, owed: bool) { + match owed { + true => self.set(arrival, Phase::Owed), + false => self.leave(arrival), + } + } + + fn leave(&mut self, arrival: Arrival) { + self.set(arrival, Phase::Held(None)); + } + + fn set(&mut self, arrival: Arrival, phase: Phase) { + let function = self.functions.get_mut(&arrival).expect("supervisor: an arrival it decided nothing about"); + function.phase = phase; + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::restart::{RESTARTS, WINDOW}; + use toyos_abi::inventory::PciAddr; + use toyos_abi::syscall::PciId; + + const EDU: PciMatch = PciMatch::new([0xff, 0x00, 0x00], Some(PciId { vendor: 0x1234, device: 0x11e8 })); + + fn row() -> Program { + Program { name: "holder".into(), drives: vec![EDU.to_string()], ..Program::default() } + } + + fn edu(arrival: u32, driven: Driven) -> Pci { + Pci { + arrival: Arrival(arrival), + at: PciAddr { segment: 0, bus: 0, dev: arrival as u8, func: 0 }, + vendor: 0x1234, + device: 0x11e8, + class: 0xff, + subclass: 0, + prog_if: 0, + driven, + } + } + + fn started(acts: &[Act<'_>]) -> Vec { + acts.iter() + .filter_map(|act| match act { + Act::Start { function, .. } => Some(function.arrival.0), + Act::GiveUp { .. } => None, + }) + .collect() + } + + /// A process per function the matcher names, none on the kernel's, none + /// on another's, and one only however often the same reading is decided. + #[test] + fn each_free_function_a_matcher_names_gets_one_process() { + let (row, now) = (row(), Instant::now()); + let rows = [(&row, EDU)]; + let mut nic = edu(9, Driven::Free); + nic.vendor = 0x1af4; + let machine = [edu(3, Driven::Free), edu(4, Driven::Free), edu(5, Driven::Kernel), edu(6, Driven::Claimed), nic]; + let mut drives = Drives::default(); + let acts = drives.decide(&rows, &machine, now); + assert_eq!(started(&acts), [3, 4]); + assert!(matches!(acts[0], Act::Start { row: r, entry, .. } if r.name == "holder" && entry == EDU)); + assert!(drives.decide(&rows, &machine, now).is_empty()); + // Read again once each process holds its function. + assert!(drives.decide(&rows, &[edu(3, Driven::Claimed), edu(4, Driven::Claimed)], now).is_empty()); + } + + /// **A refused start is not asked for again until its record changes**: + /// the same reading decided again starts nothing, however often. + #[test] + fn a_refused_start_is_not_retried_until_its_record_changes() { + let (row, now) = (row(), Instant::now()); + let rows = [(&row, EDU)]; + let machine = [edu(3, Driven::Free)]; + let mut drives = Drives::default(); + assert_eq!(started(&drives.decide(&rows, &machine, now)), [3]); + drives.refused(Arrival(3)); + for round in 0..2 * RESTARTS { + assert!(drives.decide(&rows, &machine, now).is_empty(), "round {round} started it again"); + } + // Another holder took it and gave it back: a change. + drives.decide(&rows, &[edu(3, Driven::Claimed)], now); + assert_eq!(started(&drives.decide(&rows, &machine, now)), [3]); + } + + /// Every start after a function's first spends its budget, and a function + /// that keeps refusing is given up on once, and never started again. + #[test] + fn a_function_that_keeps_refusing_is_given_up_on() { + let (row, now) = (row(), Instant::now()); + let rows = [(&row, EDU)]; + let (free, taken) = ([edu(3, Driven::Free)], [edu(3, Driven::Claimed)]); + let mut drives = Drives::default(); + let mut starts = 0; + let mut given_up = 0; + for _ in 0..4 * RESTARTS { + for act in drives.decide(&rows, &free, now) { + match act { + Act::Start { .. } => { + starts += 1; + drives.refused(Arrival(3)); + } + Act::GiveUp { function, .. } => { + assert_eq!(function.arrival, Arrival(3)); + given_up += 1; + } + } + } + drives.decide(&rows, &taken, now); + } + assert_eq!((starts, given_up), (RESTARTS + 1, 1)); + assert!(drives.decide(&rows, &free, now + 2 * WINDOW).is_empty()); + } + + /// An end owed a restart starts another at once, on the budget; one owed + /// nothing leaves the function alone like a refusal. + #[test] + fn an_ended_process_is_started_again_only_when_owed() { + let (row, now) = (row(), Instant::now()); + let rows = [(&row, EDU)]; + let machine = [edu(3, Driven::Free), edu(4, Driven::Free)]; + let mut drives = Drives::default(); + assert_eq!(started(&drives.decide(&rows, &machine, now)), [3, 4]); + drives.ended(Arrival(3), true); + drives.ended(Arrival(4), false); + assert_eq!(started(&drives.decide(&rows, &machine, now)), [3]); + assert!(drives.decide(&rows, &machine, now).is_empty()); + } + + #[test] + fn every_drives_entry_is_a_matcher_with_its_row() { + let manifest = Manifest { programs: vec![row(), Program { name: "other".into(), ..Program::default() }], ..Manifest::default() }; + let found = matchers(&manifest); + assert_eq!(found.len(), 1); + assert_eq!((found[0].0.name.as_str(), found[0].1), ("holder", EDU)); + } +} diff --git a/system/supervisor/lifecycle/src/lib.rs b/system/supervisor/lifecycle/src/lib.rs new file mode 100644 index 00000000000..ffe204e5b39 --- /dev/null +++ b/system/supervisor/lifecycle/src/lib.rs @@ -0,0 +1,10 @@ +//! The supervisor's lifecycle decisions, pure: `/system/bin/supervisor` holds +//! the handles, makes the spawns and runs the loop, and asks here what to do. +//! +//! [`restart`] is when a process that ended is started again, [`claims`] which +//! claims a start mints, and [`drives`] which PCI functions a `drives` row has +//! a process on. + +pub mod claims; +pub mod drives; +pub mod restart; diff --git a/system/supervisor/lifecycle/src/restart.rs b/system/supervisor/lifecycle/src/restart.rs new file mode 100644 index 00000000000..032a31a9e44 --- /dev/null +++ b/system/supervisor/lifecycle/src/restart.rs @@ -0,0 +1,104 @@ +//! When a process the supervisor started is started again after its end. + +use std::collections::VecDeque; +use std::time::{Duration, Instant}; + +use toyos_manifest::Program; + +/// The most starts a service, or one function's processes, spends inside +/// [`WINDOW`] after its first. +pub const RESTARTS: usize = 3; +pub const WINDOW: Duration = Duration::from_secs(10); + +/// The starts after the first that a service or a function has spent. +#[derive(Default, Debug)] +pub struct Budget { + spent: VecDeque, +} + +impl Budget { + /// Spend one start at `now`: `false`, spending nothing, once [`RESTARTS`] + /// have been spent inside [`WINDOW`]. + pub fn spend(&mut self, now: Instant) -> bool { + while self.spent.front().is_some_and(|at| now.duration_since(*at) > WINDOW) { + self.spent.pop_front(); + } + if self.spent.len() >= RESTARTS { + return false; + } + self.spent.push_back(now); + true + } +} + +/// Whether `row`'s process that ended with `exit` is owed another: the row is +/// `restart`, and it ended other than by exit 0, which a program says only +/// when it has nothing to serve. +pub fn owed(row: &Program, exit: i32) -> bool { + row.restart && exit != 0 +} + +/// What a service's end leaves the supervisor to do. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub enum AfterEnd { + /// Start it again, on the ports it kept. + Restart, + /// Close its ports: no start is owed. + Close, + /// Close its ports: a start is owed, and its budget is spent. + GiveUp, +} + +pub fn after_end(row: &Program, exit: i32, budget: &mut Budget, now: Instant) -> AfterEnd { + if !owed(row, exit) { + AfterEnd::Close + } else if budget.spend(now) { + AfterEnd::Restart + } else { + AfterEnd::GiveUp + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn row(restart: bool) -> Program { + Program { restart, serves: vec!["block".into()], ..Program::default() } + } + + #[test] + fn a_restart_row_that_ends_other_than_by_exit_0_is_started_again() { + let now = Instant::now(); + for exit in [1, -1, 101, 139] { + assert_eq!(after_end(&row(true), exit, &mut Budget::default(), now), AfterEnd::Restart, "exit {exit}"); + } + assert_eq!(after_end(&row(true), 0, &mut Budget::default(), now), AfterEnd::Close); + for exit in [0, 1, 139] { + assert_eq!(after_end(&row(false), exit, &mut Budget::default(), now), AfterEnd::Close, "exit {exit}"); + } + } + + /// The fourth end inside the window is given up on, and a window later the + /// budget is whole again. + #[test] + fn the_budget_is_restarts_inside_the_window() { + let (mut budget, now) = (Budget::default(), Instant::now()); + for _ in 0..RESTARTS { + assert_eq!(after_end(&row(true), 1, &mut budget, now), AfterEnd::Restart); + } + assert_eq!(after_end(&row(true), 1, &mut budget, now + WINDOW), AfterEnd::GiveUp); + let later = now + WINDOW + Duration::from_millis(1); + assert_eq!(after_end(&row(true), 1, &mut budget, later), AfterEnd::Restart); + } + + /// An end owed nothing spends nothing. + #[test] + fn an_end_owed_nothing_spends_no_start() { + let (mut budget, now) = (Budget::default(), Instant::now()); + for _ in 0..2 * RESTARTS { + assert_eq!(after_end(&row(true), 0, &mut budget, now), AfterEnd::Close); + } + assert!(budget.spend(now)); + } +} diff --git a/system/supervisor/src/main.rs b/system/supervisor/src/main.rs index 4884c10c797..3b2c5079dce 100644 --- a/system/supervisor/src/main.rs +++ b/system/supervisor/src/main.rs @@ -23,14 +23,21 @@ //! asks the kernel, which stops every process at once and waits for none. //! //! **A service the supervisor starts at boot outlives any one process serving it.** The supervisor -//! keeps each of its acceptors and endows a duplicate, so a swap +//! keeps each of its acceptors and endows a duplicate, so a restart or a swap //! ([`toyos_swap`]) can stop the process, start another binary holding the same //! manifest row, and leave every client's connector naming the same port — a //! connect made in the gap waits in that port's queue for the new process. A -//! service that ends with no swap owning it has its kept acceptors closed at -//! once, so a client's next connect is `ServerGone` exactly as it is for a -//! server nothing keeps. A program started through `launcher` still gets its -//! acceptor by move and is started once per boot. +//! service that ends with no swap owning it and no restart owed has its kept +//! acceptors closed at once, so a client's next connect is `ServerGone` exactly +//! as it is for a server nothing keeps. A program started through `launcher` +//! still gets its acceptor by move and is started once per boot. +//! +//! **A `drives` row has one process per PCI function it names**, each holding +//! that function's claim and started when the inventory shows the function +//! free ([`toyos_supervisor::drives`]). **Every end is readiness on the +//! process's handle**, watched on the loop's own poller: the supervisor reads +//! the inventory again after each, and what an end owes is +//! [`toyos_supervisor`]'s to decide. //! //! **The supervisor never waits on a file server it supervises from its loop alone**: //! the loop is also where a server that ended is started again. Its calls into @@ -80,11 +87,11 @@ //! neither is refused. use std::cell::{Cell, RefCell}; -use std::collections::{BTreeMap, VecDeque}; +use std::collections::BTreeMap; use std::os::toyos::process::{ChildExt, CommandExt}; use std::path::Path; use std::process::{Child, Command, Stdio}; -use std::sync::{Arc, Mutex}; +use std::rc::Rc; use std::time::{Duration, Instant}; use toyos_swap::{Refusal, Request as SwapRequest, Word}; @@ -93,7 +100,11 @@ use toyos_manifest::consent; use toyos_manifest::grants::{self, Answer, Decision, Store}; use toyos_manifest::launch::{self as authority, Authority, Session, Sessions, Target}; use toyos_manifest::package::{self, Package}; -use toyos_manifest::{Manifest, Program, Start, View}; +use toyos_manifest::device::{Entry, PciMatch}; +use toyos_manifest::{Manifest, Program, View}; +use toyos_supervisor::claims::{self, Missing, Unresolved}; +use toyos_supervisor::drives::{self, Act, Drives}; +use toyos_supervisor::restart::{self, AfterEnd, Budget}; use toyos::endow::Endowments; use toyos::fs::{Access, Grant, CAPABILITY_PREFIX}; use toyos::ipc::{self, Connection, RxStep}; @@ -112,6 +123,7 @@ use toyos_blockring::wire::{Grant as BlockGrant, Scope}; use toyos_logstream::{ Registration, Tag, ALIVE, CONSOLE, FLUSH, FLUSHED, LOGKEEPER, MAX_TAG, ORIGINS, REGISTER, RESUME, STOPPING, }; +use toyos_abi::inventory::{Pci, PciAddr, Record}; use toyos_abi::syscall::{ DeviceRequest, FileType, SyscallError, DEV_PREFIX, MAX_BADGE, PROVIDE_PREFIX, SERVE_PREFIX, SVC_LABEL, SYSCAP_LABEL, @@ -188,7 +200,8 @@ struct Minted { } /// The poll tokens for the `launcher`, `swap`, `power` and `grants` acceptors, for the -/// one connection whose swap has been answered and whose hang-up is awaited, and +/// one connection whose swap has been answered and whose hang-up is awaited, +/// for every process the supervisor watches the end of, and /// for the two of the launch waiting on an answer. A /// pending connection's token is [`TOKEN_PENDING_BASE`] plus its handle, which /// is unique among the connections the supervisor holds at once. @@ -196,13 +209,18 @@ const TOKEN_ACCEPTOR: u64 = 0; const TOKEN_SWAP_ACCEPTOR: u64 = 1; const TOKEN_HANGUP: u64 = 2; const TOKEN_POWER_ACCEPTOR: u64 = 3; -const TOKEN_WAKE: u64 = 4; +const TOKEN_ENDED: u64 = 4; const TOKEN_GRANTS_ACCEPTOR: u64 = 5; /// The parked launch's caller, and its question's connection ([`Parked`]). const TOKEN_PARKED_CALLER: u64 = 6; const TOKEN_CONSENT: u64 = 7; const TOKEN_PENDING_BASE: u64 = 8; +/// The processes whose end the supervisor watches at once: its services and +/// its `drives` processes, in what a poller holds beside the loop's own +/// watches. A `drives` process past it is not started, and says so. +const MAX_CHILDREN: usize = Poller::MAX_HANDLES as usize - TOKEN_PENDING_BASE as usize - MAX_PENDING_LAUNCHES; + /// How long a stop waits for `logkeeper` to say the log is whole. /// /// **A policy number**: the flush is a read of every ring, a write and a @@ -463,7 +481,6 @@ fn main() { .finish() .expect("supervisor: no namespace for the consent port") }); - let (wake_read, wake_write) = toyos::pipe_pair().expect("supervisor: no pipe to hear a service end"); let mut supervisor = Supervisor { system, minted: Minted { launcher, grants: grants_port }, @@ -476,12 +493,14 @@ fn main() { sessions: Sessions::default(), files, services: Vec::new(), + drivers: drives::matchers(system), + drives: Drives::default(), + instances: Vec::new(), log, - wake: wake_read, worker: Worker::start(), restarting: false, }; - supervisor.boot(&mut role_acceptors, wake_write.into_raw()); + supervisor.boot(&mut role_acceptors); // Nothing else holds a `serves` acceptor that has not been launched yet, so // the supervisor outliving its children is what keeps those ports open. It parks @@ -527,7 +546,7 @@ struct Worker { owed: u32, /// What the job still owed is, for the refusal a second one gets. owed_what: String, - /// Waits on `done` and on a service's end together. + /// Waits on `done` and on the ends of the supervisor's processes together. poller: Poller, } @@ -544,7 +563,7 @@ impl Worker { } }) .expect("supervisor: its file worker could not be started"); - Self { jobs, done, owed: 0, owed_what: String::new(), poller: Poller::new(2) } + Self { jobs, done, owed: 0, owed_what: String::new(), poller: Poller::new(1 + MAX_CHILDREN as u32) } } /// Count the jobs that have finished since last asked. @@ -595,18 +614,33 @@ struct Supervisor<'a> { files: &'static Namespace, /// What `[boot] start` named, a file server once per role. services: Vec>, + /// Every `drives` matcher, with its row. + drivers: Vec<(&'static Program, PciMatch)>, + /// Which functions have a `drives` process, and which are owed one. + drives: Drives, + /// The `drives` processes running. + instances: Vec, /// Where a service the supervisor (re)starts writes its stdout and stderr, whichever /// process ends up holding its row. log: Log, - /// Readable when a `restart` service has ended and is owed a new process. - wake: Pipe, /// Where the supervisor's calls into the file servers are made. worker: Worker, - /// Inside [`Supervisor::restart_ended`]: a file call made there leaves the wake - /// for the loop, which is the one caller that acts on it. + /// Inside [`Supervisor::ended`]: a file call made there leaves an end for + /// the loop, which is the one caller that acts on it. restarting: bool, } +/// One `drives` process, and the function it holds. +struct Instance { + row: &'static Program, + function: Pci, + child: Child, +} + +/// A service's kept acceptors, shared with what [`Grants`] mints on them. +/// Empty once its ports have closed for good. +type Ports = Rc>>; + /// One program the supervisor started at boot, and the ports it serves. struct Service<'a> { program: &'a Program, @@ -620,57 +654,29 @@ struct Service<'a> { /// The row's devices the running process was endowed: what a process /// started in its place is owed. devices: Vec, - kept: Arc>, + ports: Ports, + /// A swap owns the service: its process ending is expected, the + /// supervisor does not watch it, and the ports stay open for the next one. + swapping: bool, /// The machine's session under this service's own share, which every /// start of it spends: one started again holds no more of a file server. session: Session, - /// When each recent start after an end happened, for [`toyos_manifest::RESTARTS`]. - restarts: VecDeque, -} - -/// What a service's waiter thread shares with the loop. -struct Kept { - /// The acceptors of every name the service serves. Empty once the service - /// has ended with no swap owning it and no restart owed: the port is then - /// closed for good. - acceptors: Vec<(String, Acceptor)>, - /// Counts the service's starts, so a waiter answers only for its own. - generation: u64, - /// A swap owns the service: its process ending is expected and the port - /// stays open for the next one. - swapping: bool, - /// The row, which says what its end is owed ([`Program::supervised`]). - row: Program, - /// It ended, and the loop owes it a new process on the same ports. - ended: bool, - /// The write end of [`Supervisor::wake`]. - wake: toyos::RawHandle, + /// The starts after an end it has spent. + budget: Budget, } impl<'a> Service<'a> { - fn new( - program: &'a Program, - role: Option<&'a str>, - acceptors: Vec<(String, Acceptor)>, - session: Session, - wake: toyos::RawHandle, - ) -> Self { + fn new(program: &'a Program, role: Option<&'a str>, acceptors: Vec<(String, Acceptor)>, session: Session) -> Self { Self { program, role, path: program.path.clone(), child: None, devices: Vec::new(), - kept: Arc::new(Mutex::new(Kept { - acceptors, - generation: 0, - swapping: false, - row: program.clone(), - ended: false, - wake, - })), + ports: Rc::new(RefCell::new(acceptors)), + swapping: false, session, - restarts: VecDeque::new(), + budget: Budget::default(), } } @@ -700,14 +706,9 @@ impl<'a> Service<'a> { std::io::Error::other(format!("{path} no longer holds what was installed: {why}")) })?; } - let mut kept = self.kept.lock().expect("supervisor: a service's state is poisoned"); - // Every start after the first follows a process of this service - // that the supervisor stopped or saw end. - let served = match kept.generation { - 0 => Served::Keep(&kept.acceptors), - _ => Served::Restart { acceptors: &kept.acceptors, owed }, - }; let storage = storage_endowment(self.program, self.role, syscap, grants)?; + let ports = self.ports.borrow(); + let served = Served::Keep { acceptors: &ports, owed }; let (child, devices) = start( Command::new(path), self.program, @@ -721,17 +722,7 @@ impl<'a> Service<'a> { (minted, self.session), Output::Boot(log), )?; - kept.generation += 1; - kept.ended = false; - if self.program.supervised(Start::Spawned) { - let process = toyos_abi::syscall::dup(toyos::RawHandle(child.as_raw_handle())) - .unwrap_or_else(|e| panic!("supervisor: {}'s process handle: {e:?}", self.program.name)); - let (shared, generation) = (Arc::clone(&self.kept), kept.generation); - std::thread::Builder::new() - .name(format!("wait-{}", self.program.name)) - .spawn(move || close_when_it_ends(&shared, generation, process)) - .expect("supervisor: a service's waiter could not be started"); - } + drop(ports); let pid = child.id(); self.child = Some(child); self.path = path.to_string(); @@ -753,11 +744,12 @@ impl<'a> Service<'a> { } } -/// Why a service did not start. +/// Why a process did not start. enum StartError { - /// The partition its file-server role is on was refused, so the role is - /// absent: its ports close and nothing serves its paths from memory. - Partition(String), + /// This machine refuses its row a start — the partition its file-server + /// role is on, a `devices` matcher naming two functions — so the row is + /// absent this boot: its ports close, and nothing serves them from memory. + Refused(String), Other(std::io::Error), } @@ -770,30 +762,12 @@ impl From for StartError { impl std::fmt::Display for StartError { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { match self { - Self::Partition(why) => f.write_str(why), + Self::Refused(why) => f.write_str(why), Self::Other(e) => e.fmt(f), } } } -/// Wait for one start of a service to end, and close its ports if nothing was -/// expecting it to, or wake the loop to start it again if its row says so. -fn close_when_it_ends(kept: &Mutex, generation: u64, process: toyos::RawHandle) { - let exit = toyos_abi::syscall::process_wait(process) - .unwrap_or_else(|e| panic!("supervisor: a service's own process handle refused its wait: {e:?}")); - toyos_abi::syscall::close(process); - let mut kept = kept.lock().expect("supervisor: a service's state is poisoned"); - if kept.generation == generation && !kept.swapping { - if kept.row.supervised(Start::Ended(exit)) { - kept.ended = true; - // A full pipe is a wake already pending, which is all this is. - let _ = toyos_abi::syscall::write_nonblock(kept.wake, &[1]); - } else { - kept.acceptors.clear(); - } - } -} - /// A swap the supervisor has answered and not finished. struct Flight { /// Index into [`Supervisor::services`]. @@ -835,8 +809,9 @@ impl<'a> Supervisor<'a> { /// **Every service holds its acceptors before any is started**, so what a /// start mints on another service's port — a role's directories, a /// partition, `netstack` — does not depend on the order `[boot] start` - /// lists them in. - fn boot(&mut self, role_acceptors: &mut BTreeMap<&str, Acceptor>, wake: toyos::RawHandle) { + /// lists them in. The `drives` processes start last, on what the + /// inventory says then. + fn boot(&mut self, role_acceptors: &mut BTreeMap<&str, Acceptor>) { let system = self.system; let rank = |name: &&String| match system.program(name) { Some(p) if is_block(p) => 0, @@ -872,16 +847,16 @@ impl<'a> Supervisor<'a> { vec![(format!("{CAPABILITY_PREFIX}{role}"), acceptor)] } }; - let service = Service::new(program, role, kept, self.sessions.machine(), wake); + let service = Service::new(program, role, kept, self.sessions.machine()); if let Some(role) = role { - self.grants.roles.push((role, Arc::clone(&service.kept))); + self.grants.roles.push((role, Rc::clone(&service.ports))); } if is_block(program) { assert!(self.grants.block.is_none(), "supervisor: two rows start a block service"); - self.grants.block = Some(Arc::clone(&service.kept)); + self.grants.block = Some(Rc::clone(&service.ports)); } if program.serves.iter().any(|name| name == NETSTACK) { - self.grants.netstack = Some(Arc::clone(&service.kept)); + self.grants.netstack = Some(Rc::clone(&service.ports)); } self.services.push(service); } @@ -899,8 +874,8 @@ impl<'a> Supervisor<'a> { service.spawn(&program.path, &[], system, self.syscap, &self.connectors, &self.grants, &self.minted, &mut self.log); match started { Ok(_) => {} - Err(StartError::Partition(why)) => { - service.kept.lock().expect("supervisor: a service's state is poisoned").acceptors.clear(); + Err(StartError::Refused(why)) => { + service.ports.borrow_mut().clear(); say!("supervisor: {} did not start ({why}); its ports are closed this boot", service.label()); } Err(e) => panic!("supervisor: cannot start {}: {e}", program.name), @@ -910,6 +885,55 @@ impl<'a> Supervisor<'a> { self.make_session_home(); } self.load_grants(); + self.drive(); + } + + /// Start a `drives` process on each function the inventory says is owed + /// one, and say which functions are given up on. + fn drive(&mut self) { + let records = match pci_records(self.syscap) { + Ok(records) => records, + Err(why) => { + say!("supervisor: {why}, so no `drives` process starts on it"); + return; + } + }; + for act in self.drives.decide(&self.drivers, &records, Instant::now()) { + match act { + Act::Start { row, entry, function } => match self.start_instance(row, entry, function) { + Ok(child) => self.instances.push(Instance { row, function, child }), + Err(why) => { + say!("supervisor: {} was not started on PCI {}: {why}", row.name, pci_at(function.at)); + self.drives.refused(function.arrival); + } + }, + Act::GiveUp { row, function } => say!( + "supervisor: {} was started on PCI {} {} times in {} s and is given up on", + row.name, + pci_at(function.at), + restart::RESTARTS + 1, + restart::WINDOW.as_secs() + ), + } + } + } + + /// One `drives` process of `row`, holding `function`'s claim under + /// `entry`'s spelling. + fn start_instance(&mut self, row: &'static Program, entry: PciMatch, function: Pci) -> Result { + if self.services.len() + self.instances.len() >= MAX_CHILDREN { + return Err(format!("the supervisor watches {MAX_CHILDREN} processes already")); + } + let name = entry.to_string(); + let claim = self.syscap.claim_function::(function.arrival).map_err(|e| refused(&name, e))?; + self.make_home(row); + let extra = Extra { claims: vec![(format!("{DEV_PREFIX}{name}"), claim)], ..Extra::default() }; + let launcher = (&self.minted, self.sessions.machine()); + let served = Served::Keep { acceptors: &[], owed: &[] }; + let output = Output::Boot(&mut self.log); + start(Command::new(&row.path), row, self.system, self.syscap, served, &self.connectors, &self.grants, &[], extra, launcher, output) + .map(|(child, _)| child) + .map_err(|e| e.to_string()) } /// [`grants::STORE`], read on the worker once DATA's server runs. A store @@ -1053,13 +1077,13 @@ impl<'a> Supervisor<'a> { /// (`toyos_update::slots`): minted here, so the slot this boot runs is /// never among what is endowed. A machine with none says so and the /// program starts holding nothing, which it refuses by name. - fn slot_storage(&mut self, program: &Program) -> Storage { + fn slot_storage(&mut self, program: &Program) -> Extra { if !program.slots { - return Storage::default(); + return Extra::default(); } self.slot_grant().unwrap_or_else(|why| { say!("supervisor: {}: no slot to grant: {why}", program.name); - Storage::default() + Extra::default() }) } @@ -1069,8 +1093,8 @@ impl<'a> Supervisor<'a> { /// its two partitions are reached only as `toyos_update::slots::grant` /// admits them: claimed on a disk the kernel drives, and on one the block /// service serves, through connectors minted for each. - fn slot_grant(&mut self) -> Result { - use toyos_abi::inventory::{Record, Role}; + fn slot_grant(&mut self) -> Result { + use toyos_abi::inventory::Role; use toyos_update::slots::{BOOT_LABEL, ROOT_LABEL, TABLE_LABEL}; let records = inventory(self.syscap)?; let running = loaded(&records, Role::Root).ok_or("the loader named no ROOT")?; @@ -1106,19 +1130,18 @@ impl<'a> Supervisor<'a> { "supervisor: the idle slot is {}, granted with the slot table read through the block service", idle.letter() ); - Ok(Storage { ports, ..Storage::default() }) + Ok(Extra { ports, ..Extra::default() }) } /// `work`, a call into the file servers or the block service, made on - /// [`Worker`], and its answer — with every server that ends meanwhile - /// started again, so a call its end left waiting in the port's queue goes - /// on to the new process. + /// [`Worker`], and its answer — with every end meanwhile acted on, so a + /// call a server's end left waiting in the port's queue goes on to the + /// process started in its place. /// `Err` is the call still unanswered at [`FILES_BOUND`], which the worker /// goes on waiting for alone, or the worker still waiting on an earlier /// one. fn files(&mut self, what: &str, work: impl FnOnce() -> T + Send + 'static) -> Result { const DONE: u64 = 0; - const WAKE: u64 = 1; self.worker.settle(); if self.worker.owed > 0 { return Err(format!("the supervisor's file worker still waits on {}", self.worker.owed_what)); @@ -1135,15 +1158,13 @@ impl<'a> Supervisor<'a> { loop { self.worker.poller.watch(&self.worker.done, READABLE, DONE); if !self.restarting { - self.worker.poller.watch(&self.wake, READABLE, WAKE); + self.watch_children(&self.worker.poller); } let left = deadline.saturating_duration_since(Instant::now()); let mut woken = false; - self.worker.poller.wait(1, left.as_nanos() as u64, |token| woken |= token == WAKE); + self.worker.poller.wait(1, left.as_nanos() as u64, |token| woken |= token == TOKEN_ENDED); if woken { - let mut sink = [0u8; 64]; - while matches!(self.wake.read_nonblock(&mut sink), Ok(n) if n > 0) {} - self.restart_ended(); + self.ended(); } self.worker.settle(); if self.worker.owed == 0 { @@ -1169,7 +1190,7 @@ impl<'a> Supervisor<'a> { /// service it has just killed to finish ending, which is the kernel's /// teardown and no client's. fn serve_forever(&mut self, swap: &Acceptor, power: &Acceptor) -> ! { - let poller = Poller::new(8 + MAX_PENDING_LAUNCHES as u32); + let poller = Poller::new(Poller::MAX_HANDLES); let mut pending: Vec = Vec::new(); let mut flight: Option = None; let mut parked: Option = None; @@ -1179,7 +1200,7 @@ impl<'a> Supervisor<'a> { poller.watch(&self.minted.grants, READABLE, TOKEN_GRANTS_ACCEPTOR); poller.watch(swap, READABLE, TOKEN_SWAP_ACCEPTOR); poller.watch(power, READABLE, TOKEN_POWER_ACCEPTOR); - poller.watch(&self.wake, READABLE, TOKEN_WAKE); + self.watch_children(&poller); if let Some(Flight { phase: Phase::Answered { conn, .. }, .. }) = &flight { poller.watch(conn, READABLE, TOKEN_HANGUP); } @@ -1315,10 +1336,8 @@ impl<'a> Supervisor<'a> { let (caller_ready, consent_ready) = (ready.contains(&TOKEN_PARKED_CALLER), ready.contains(&TOKEN_CONSENT)); parked = parked.and_then(|asked| self.advance_parked(asked, caller_ready, consent_ready)); - if ready.contains(&TOKEN_WAKE) { - let mut sink = [0u8; 64]; - while matches!(self.wake.read_nonblock(&mut sink), Ok(n) if n > 0) {} - self.restart_ended(); + if ready.contains(&TOKEN_ENDED) { + self.ended(); } } } @@ -1455,7 +1474,7 @@ impl<'a> Supervisor<'a> { Word::Stopping, &format!("pid {} ({previous})", service.pid().map_or(0, |p| p)), ); - service.kept.lock().expect("supervisor: a service's state is poisoned").swapping = true; + service.swapping = true; if let Some(mut old) = service.child.take() { // Killed and then waited for, because the kernel publishes a // process's end only after its handle table is closed: once the @@ -1493,17 +1512,12 @@ impl<'a> Supervisor<'a> { fn end_probation(&mut self, index: usize, path: &str, previous: &str, owed: &[String]) { let service = &mut self.services[index]; let name = service.program.name.clone(); - let status = { - // Asked under the lock the waiter takes, so an end after this - // answer is the waiter's to act on and an end before it is this - // function's. - let mut kept = service.kept.lock().expect("supervisor: a service's state is poisoned"); - let status = service.child.as_mut().map(|c| c.try_wait()); - if matches!(status, Some(Ok(None))) { - kept.swapping = false; - } - status - }; + // The loop watches its end from here, and an end before this answer + // is this function's. + let status = service.child.as_mut().map(|c| c.try_wait()); + if matches!(status, Some(Ok(None))) { + service.swapping = false; + } match status { Some(Ok(None)) => { swapped( @@ -1599,53 +1613,67 @@ impl<'a> Supervisor<'a> { } } - /// Start again every `restart` service that ended, on the ports it kept, - /// unless it has ended [`toyos_manifest::RESTARTS`] times inside - /// [`toyos_manifest::RESTART_WINDOW_SECS`]: then its ports close, and a - /// client's next connection is answered `Gone`. - fn restart_ended(&mut self) { + /// Watch the end of every process the supervisor started and has not seen + /// end, but a swapping service's: its swap watches that. + fn watch_children(&self, poller: &Poller) { + let services = self.services.iter().filter(|service| !service.swapping).filter_map(|service| service.child.as_ref()); + for child in services.chain(self.instances.iter().map(|instance| &instance.child)) { + poller.watch_raw(toyos::RawHandle(child.as_raw_handle()), READABLE, TOKEN_ENDED); + } + } + + /// Act on every end since the last: a service is started again on the + /// ports it kept, or has them closed and its clients answered `Gone` + /// ([`restart::after_end`]); a `drives` process's function is owed + /// another or left alone ([`Drives::ended`]); and the inventory is read + /// again. + fn ended(&mut self) { self.restarting = true; - let window = Duration::from_secs(toyos_manifest::RESTART_WINDOW_SECS); + let mut any = false; for index in 0..self.services.len() { - let ended = self.services[index].kept.lock().expect("supervisor: a service's state is poisoned").ended; - if !ended { - continue; - } let service = &mut self.services[index]; - let label = service.label(); - let now = Instant::now(); - while service.restarts.front().is_some_and(|at| now.duration_since(*at) > window) { - service.restarts.pop_front(); - } - let pid = service.pid().map_or(0, |p| p); - if let Some(mut old) = service.child.take() { - let _ = old.wait(); - } - if service.restarts.len() as u32 >= toyos_manifest::RESTARTS { - let mut kept = service.kept.lock().expect("supervisor: a service's state is poisoned"); - kept.ended = false; - kept.acceptors.clear(); - say!( - "supervisor: {label} ended {} times in {} s; its ports are closed, its clients answered Gone, and a program started from now on is given none of its directories", - toyos_manifest::RESTARTS + 1, - toyos_manifest::RESTART_WINDOW_SECS - ); - continue; - } - service.restarts.push_back(now); - let (path, owed, program) = (service.path.clone(), service.devices.clone(), service.program); - self.make_home(program); - let service = &mut self.services[index]; - match service.spawn(&path, &owed, self.system, self.syscap, &self.connectors, &self.grants, &self.minted, &mut self.log) { - Ok(new) => say!("supervisor: {label} (pid {pid}) ended; started again as pid {new}"), - Err(e) => { - let mut kept = service.kept.lock().expect("supervisor: a service's state is poisoned"); - kept.ended = false; - kept.acceptors.clear(); - say!("supervisor: {label} ended and would not start again ({e}); its ports are closed"); + let Some(exit) = service.child.as_ref().filter(|_| !service.swapping).and_then(exit_of) else { continue }; + any = true; + let (label, pid) = (service.label(), service.pid().unwrap_or(0)); + service.child = None; + match restart::after_end(service.program, exit, &mut service.budget, Instant::now()) { + AfterEnd::Close => service.ports.borrow_mut().clear(), + AfterEnd::GiveUp => { + service.ports.borrow_mut().clear(); + say!( + "supervisor: {label} ended {} times in {} s; its ports are closed, its clients answered Gone, and a program started from now on is given none of its directories", + restart::RESTARTS + 1, + restart::WINDOW.as_secs() + ); + } + AfterEnd::Restart => { + let (path, owed, program) = (service.path.clone(), service.devices.clone(), service.program); + self.make_home(program); + let service = &mut self.services[index]; + match service.spawn(&path, &owed, self.system, self.syscap, &self.connectors, &self.grants, &self.minted, &mut self.log) { + Ok(new) => say!("supervisor: {label} (pid {pid}) ended; started again as pid {new}"), + Err(e) => { + service.ports.borrow_mut().clear(); + say!("supervisor: {label} ended and would not start again ({e}); its ports are closed"); + } + } } } } + let mut at = 0; + while at < self.instances.len() { + let Some(exit) = exit_of(&self.instances[at].child) else { + at += 1; + continue; + }; + any = true; + let Instance { row, function, .. } = self.instances.swap_remove(at); + say!("supervisor: {} on PCI {} ended with {exit}", row.name, pci_at(function.at)); + self.drives.ended(function.arrival, restart::owed(row, exit)); + } + if any { + self.drive(); + } self.restarting = false; } @@ -1658,13 +1686,12 @@ impl<'a> Supervisor<'a> { let name = service.program.name.clone(); match service.spawn(previous, owed, self.system, self.syscap, &self.connectors, &self.grants, &self.minted, &mut self.log) { Ok(pid) => { - service.kept.lock().expect("supervisor: a service's state is poisoned").swapping = false; + service.swapping = false; swapped(&name, Word::Restored, &format!("{previous} as pid {pid}")); } Err(e) => { - let mut kept = service.kept.lock().expect("supervisor: a service's state is poisoned"); - kept.swapping = false; - kept.acceptors.clear(); + service.swapping = false; + service.ports.borrow_mut().clear(); swapped( &name, Word::Gone, @@ -2127,7 +2154,7 @@ impl Supervisor<'_> { // here answers it: the command is prepared, so its spawn calls no // file server, and every refusal `start` makes before the spawn is // `Other`. - Err(e) if e.kind() == std::io::ErrorKind::BrokenPipe && placed => { + Err(StartError::Other(e)) if e.kind() == std::io::ErrorKind::BrokenPipe && placed => { say!("supervisor: launcher: {} was not started: the place it names is ending", program.name); let _ = conn.try_signal(launch::MSG_GONE); } @@ -2295,7 +2322,7 @@ fn claimed_slots( syscap: &SysCap, parts: &[toyos_abi::inventory::Partition], running: toyos_abi::inventory::Partition, -) -> Result { +) -> Result { use toyos_abi::inventory::Partition; let tables: Vec<&Partition> = parts .iter() @@ -2328,7 +2355,7 @@ fn claimed_slots( (toyos_update::slots::BOOT_LABEL.to_string(), boot), (toyos_update::slots::ROOT_LABEL.to_string(), root), ]; - Ok(Storage { claims, ..Storage::default() }) + Ok(Extra { claims, ..Extra::default() }) } /// The types a slot's two partitions carry. @@ -2390,9 +2417,7 @@ fn refused(name: &str, why: SyscallError) -> String { match why { SyscallError::NotFound => format!("no {name} on this machine"), SyscallError::AlreadyExists => format!("{name} is already claimed"), - SyscallError::InvalidArgument => { - format!("{name} names more than one device on this machine") - } + SyscallError::InvalidArgument => format!("{name} names two partitions on this machine"), SyscallError::PermissionDenied => { format!("{name} is driven by the kernel and cannot be claimed") } @@ -2405,18 +2430,40 @@ fn refused(name: &str, why: SyscallError) -> String { } } +/// What the supervisor says about an entry a start holds no claim for. +fn missing(name: &str, why: &Missing) -> String { + let functions = |at: &[PciAddr]| at.iter().map(|at| format!("PCI {}", pci_at(*at))).collect::>().join(" and "); + match why { + Missing::Unresolved(Unresolved::Absent) => format!("no {name} on this machine"), + Missing::Unresolved(Unresolved::KernelDriven(at)) => { + format!("{name} is {}, which the kernel drives and which cannot be claimed", functions(at)) + } + Missing::Unresolved(Unresolved::Ambiguous(at)) => { + format!("{name} names {}, and its row has one process to drive one", functions(at)) + } + Missing::Refused(e) => refused(name, *e), + } +} + +/// A function's address as the kernel's `pcidev:` lines spell it. +fn pci_at(at: PciAddr) -> String { + format!("{:02x}:{:02x}.{}", at.bus, at.dev, at.func) +} + +/// The exit code of `child` once it has ended. +fn exit_of(child: &Child) -> Option { + toyos_abi::syscall::process_wait_nonblock(toyos::RawHandle(child.as_raw_handle())).ok() +} + /// Where the acceptors of the names a started program serves come from. enum Served<'m, 'a> { /// The machine-wide map a launch takes from, by move: such a program is /// started once per boot. Move(&'m mut BTreeMap<&'a str, Acceptor>), - /// A boot service's own, which the supervisor keeps and endows a duplicate of. - Keep(&'m [(String, Acceptor)]), - /// The same, for a service whose process has ended, which gave back every - /// claim it held: `owed` names them. A start that cannot mint one of them - /// is refused, never a service running on without a device the process - /// before it held. - Restart { acceptors: &'m [(String, Acceptor)], owed: &'m [String] }, + /// A boot service's own, which the supervisor keeps and endows a + /// duplicate of, and the claims the process it follows held + /// ([`claims::goes_on_without`]). + Keep { acceptors: &'m [(String, Acceptor)], owed: &'m [String] }, } /// Build one program's authority and spawn it holding exactly that. @@ -2437,11 +2484,11 @@ fn start<'a>( connectors: &BTreeMap<&str, Connector>, grants: &Grants<'_>, extras: &[(String, Connector)], - storage: Storage, + extra: Extra, launcher: (&Minted, Session), output: Output<'_>, -) -> std::io::Result<(Child, Vec)> { - let Storage { args, claims, ports } = storage; +) -> Result<(Child, Vec), StartError> { + let Extra { args, claims, ports } = extra; // A storage row's own arguments first: a file server's role leads its argv. command.args(&args); command.args(&program.args); @@ -2501,7 +2548,7 @@ fn start<'a>( // send one it narrowed `DUP` away from. That is a refused launch, not // the supervisor's bug — and the supervisor is the one process the machine cannot lose. let Ok(handed) = connector.duplicate() else { - return Err(std::io::Error::other("a provided connector cannot be duplicated")); + return Err(std::io::Error::other("a provided connector cannot be duplicated").into()); }; let raw = handed.into_raw(); command.endow(&format!("{PROVIDE_PREFIX}{name}"), raw.0); @@ -2510,8 +2557,8 @@ fn start<'a>( let mut given_back = None; let owed: &[String] = match &served { - Served::Restart { owed, .. } => owed, - Served::Move(_) | Served::Keep(_) => &[], + Served::Keep { owed, .. } => owed, + Served::Move(_) => &[], }; match served { Served::Move(acceptors) => { @@ -2532,12 +2579,12 @@ fn start<'a>( } given_back = Some(acceptors); } - Served::Keep(kept) | Served::Restart { acceptors: kept, .. } => { + Served::Keep { acceptors: kept, .. } => { // Every acceptor the service keeps: its row's `serves`, or the // directories of a file server's role. None left is a service // whose ports closed for good. if kept.is_empty() && (!program.serves.is_empty() || !program.roles.is_empty()) { - return Err(std::io::Error::other("its ports are closed for good")); + return Err(std::io::Error::other("its ports are closed for good").into()); } for (name, acceptor) in kept { let handed = toyos_abi::syscall::dup(acceptor.as_handle()).map_err(|e| { @@ -2550,55 +2597,51 @@ fn start<'a>( } let mut endowed: Vec = Vec::new(); - let mut unpaid = None; + // Read at the first PCI matcher, and once. + let mut functions: Option> = None; for name in &program.devices { - // The build system already refused a config this cannot parse - // (`names_only_real_capabilities`), so a failure here is an image built - // against a different ABI rather than somebody's typo. - let request = DeviceRequest::parse(name) - .unwrap_or_else(|| panic!("supervisor: `{name}` is not a device this ABI has")); - // A device this machine does not have is not endowed, and the supervisor says - // which. - // + // The build refuses to render an entry that names no device, so a + // failure here is an image whose halves disagree. + let entry = Entry::parse(name).unwrap_or_else(|| panic!("supervisor: `{name}` names no device")); + let request = match entry { + Entry::Fixed(request) => Ok(request), + Entry::Pci(matcher) => { + let records = match &mut functions { + Some(records) => records, + None => functions.insert(pci_records(syscap).map_err(StartError::Refused)?), + }; + claims::function(&matcher, records).map(DeviceRequest::Function).map_err(Missing::Unresolved) + } + }; + let minted = request.and_then(|request| claim(syscap, request).map_err(Missing::Refused)); // The label is the manifest's own spelling, which is exactly what the // claimant looks the claim up by: one string, written once. - let minted = match request { - DeviceRequest::Class(class) => syscap.claim::(class), - DeviceRequest::Pci(id) => syscap.claim_pci::(id), - DeviceRequest::Partition(name) => syscap.claim_partition::(name), - DeviceRequest::Isa(set) => syscap.claim_isa::(set), - }; - match minted { + let why = match minted { Ok(claim) => { let raw = claim.into_raw(); command.endow(&format!("{DEV_PREFIX}{name}"), raw.0); held.0.push(raw); endowed.push(name.clone()); + continue; } - Err(e) if owed.contains(name) => { - unpaid = Some(std::io::Error::other(format!( - "{}, and the process it replaces held it", - refused(name, e) - ))); - break; - } - // Each refusal keeps the word the kernel gave it. "This machine - // has none" is a configuration and every other answer is a fault, - // and one sentence for all six sends whoever reads the line looking - // in the wrong place. - Err(e) => { - say!("supervisor: {}: {}", program.name, refused(name, e)); - // A block service is told, so the partitions on a controller - // the machine has are refused and never taken for none. - if e != SyscallError::NotFound && is_block(program) { - command.args(["--claim-refused", name.as_str()]); - } - } + Err(why) => why, + }; + // Nothing was spawned, so everything minted goes back with `held`. + if !claims::goes_on_without(name, &why, owed) { + let held_before = if owed.contains(name) { ", and the process it replaces held it" } else { "" }; + return Err(StartError::Refused(format!("{}{held_before}", missing(name, &why)))); + } + // Each refusal keeps the word the kernel gave it. "This machine has + // none" is a configuration and every other answer is a fault, and one + // sentence for all of them sends whoever reads the line looking in + // the wrong place. + say!("supervisor: {}: {}", program.name, missing(name, &why)); + // A block service is told, so the partitions on a controller the + // machine has are refused and never taken for none. + let absent = matches!(why, Missing::Unresolved(Unresolved::Absent) | Missing::Refused(SyscallError::NotFound)); + if !absent && is_block(program) { + command.args(["--claim-refused", name.as_str()]); } - } - // Nothing was spawned, so everything minted goes back with `held`. - if let Some(unpaid) = unpaid { - return Err(unpaid); } if !program.syscap.is_empty() { @@ -2703,7 +2746,7 @@ fn start<'a>( if let Some(acceptor) = origins { log.acceptor = Some(acceptor); } - Err(e) + Err(e.into()) } } } @@ -2789,12 +2832,12 @@ fn build_namespace( /// row and session of the program that holds it. struct Grants<'a> { /// Each role's service, whose kept acceptor is the role's port. - roles: Vec<(&'a str, Arc>)>, + roles: Vec<(&'a str, Ports)>, /// The block service the supervisor started, whose kept acceptor is /// [`toyos_blockring::PORT`]. - block: Option>>, + block: Option, /// The service the supervisor started serving [`NETSTACK`]. - netstack: Option>>, + netstack: Option, } impl Grants<'_> { @@ -2802,9 +2845,9 @@ impl Grants<'_> { /// on a machine whose boot starts no block service, and `Err` once its /// port has closed for good. fn partitions(&self, grant: BlockGrant) -> Result, String> { - let Some(kept) = &self.block else { return Ok(None) }; - let kept = kept.lock().expect("supervisor: a service's state is poisoned"); - let Some((_, acceptor)) = kept.acceptors.iter().find(|(name, _)| name == toyos_blockring::PORT) else { + let Some(ports) = &self.block else { return Ok(None) }; + let ports = ports.borrow(); + let Some((_, acceptor)) = ports.iter().find(|(name, _)| name == toyos_blockring::PORT) else { return Err("the block service's port has closed for good".into()); }; acceptor @@ -2823,13 +2866,12 @@ impl Grants<'_> { if !program.receives.iter().any(|name| name == NETSTACK) { return None; } - let kept = self + let ports = self .netstack .as_ref() .unwrap_or_else(|| panic!("supervisor: {} receives `{NETSTACK}`, which no started row serves", program.name)) - .lock() - .expect("supervisor: a service's state is poisoned"); - let (_, acceptor) = kept.acceptors.iter().find(|(name, _)| name == NETSTACK)?; + .borrow(); + let (_, acceptor) = ports.iter().find(|(name, _)| name == NETSTACK)?; let badge = Authority { row: program.name.clone(), session }.encode(); let connector = acceptor .mint(&badge) @@ -2840,17 +2882,16 @@ impl Grants<'_> { /// The directory capabilities `program` is endowed for one start in /// `session`, by namespace name: its row's view (`Program::view`), but /// that a storage row sees none, since a file server resolving a path of - /// its own through itself waits for ever. Asked before anything is locked, - /// since a storage row's start holds its own kept state. + /// its own through itself waits for ever. fn view(&self, program: &Program, session: Session) -> Vec<(String, Connector)> { if is_storage(program) { return Vec::new(); } let wanted = program.view(); let mut view = Vec::new(); - for (role, kept) in &self.roles { - let kept = kept.lock().expect("supervisor: a service's state is poisoned"); - let Some((_, acceptor)) = kept.acceptors.first() else { continue }; + for (role, ports) in &self.roles { + let ports = ports.borrow(); + let Some((_, acceptor)) = ports.first() else { continue }; for dir in wanted.iter().filter(|dir| dir.role == *role) { view.push(mint(acceptor, session.share(), dir)); } @@ -2951,12 +2992,12 @@ fn swapped(service: &str, word: Word, detail: &str) { say!("{}", toyos_swap::said(service, word, detail)); } -/// What a process is started with beside its row for the partitions it -/// reaches: its arguments, the claims endowed by label on the partitions a -/// disk the kernel drives carries, and the connectors in its namespace by -/// name to those the block service serves, each minted for what it reaches. +/// What a process is started with beside its row: arguments, claims endowed +/// by label — on the partitions a disk the kernel drives carries, or a +/// `drives` process's function — and connectors in its namespace by name to +/// the partitions the block service serves, each minted for what it reaches. #[derive(Default)] -struct Storage { +struct Extra { args: Vec, claims: Vec<(String, toyos::Device)>, ports: Vec<(String, Connector)>, @@ -2969,6 +3010,22 @@ fn inventory(syscap: &SysCap) -> Result, Strin .map_err(|why| why.to_string()) } +/// Every PCI function the kernel's inventory answers. +fn pci_records(syscap: &SysCap) -> Result, String> { + let records = inventory(syscap).map_err(|why| format!("the inventory was not read: {why}"))?; + Ok(records.into_iter().filter_map(|r| if let Record::Pci(pci) = r { Some(pci) } else { None }).collect()) +} + +/// The claim `request` asks for. +fn claim(syscap: &SysCap, request: DeviceRequest) -> Result { + match request { + DeviceRequest::Class(class) => syscap.claim(class), + DeviceRequest::Function(arrival) => syscap.claim_function(arrival), + DeviceRequest::Partition(guid) => syscap.claim_partition(guid), + DeviceRequest::Isa(set) => syscap.claim_isa(set), + } +} + /// The unique GUID the loader named for `role`. fn loaded(records: &[toyos_abi::inventory::Record], role: toyos_abi::inventory::Role) -> Option<[u8; 16]> { records.iter().find_map(|r| match r { @@ -2999,9 +3056,9 @@ fn storage_endowment( role: Option<&str>, syscap: &SysCap, grants: &Grants<'_>, -) -> Result { +) -> Result { use toyos_abi::inventory::{Record, Role}; - let mut storage = Storage::default(); + let mut storage = Extra::default(); if !is_block(program) && role.is_none() { return Ok(storage); } @@ -3031,7 +3088,7 @@ fn storage_endowment( "log" | "boot" => { let which = if role == "log" { Role::Log } else { Role::Boot }; let Some(guid) = loaded(&records, which) else { - return Err(StartError::Partition(format!("the loader named no `{role}` partition"))); + return Err(StartError::Refused(format!("the loader named no `{role}` partition"))); }; let grant = BlockGrant { scope: Scope::Unique(guid), writes: role == "log" }; (on_kernel_disk(&|p| p.unique_guid == guid), Some(guid), grant) @@ -3041,10 +3098,10 @@ fn storage_endowment( for guid in &kernel { let request = toyos_abi::syscall::DeviceRequest::Partition(toyos_abi::part::PartGuid(*guid)); let mut buf = [0u8; toyos_abi::syscall::DeviceRequest::MAX_NAME]; - let name = request.write_name(&mut buf).to_string(); + let name = request.write_name(&mut buf).expect("a partition has a name").to_string(); match syscap.claim_partition::(toyos_abi::part::PartGuid(*guid)) { Ok(claim) => storage.claims.push((format!("{DEV_PREFIX}{name}"), claim)), - Err(e) => return Err(StartError::Partition(refused(&name, e))), + Err(e) => return Err(StartError::Refused(refused(&name, e))), } } // DATA is counted over both; a named partition no claim reached is the @@ -3053,7 +3110,7 @@ fn storage_endowment( if let Some(guid) = named { storage.args.push(guid_text(guid)); } - if let Some(port) = grants.partitions(served).map_err(StartError::Partition)? { + if let Some(port) = grants.partitions(served).map_err(StartError::Refused)? { storage.ports.push((toyos_blockring::PORT.to_string(), port)); } } diff --git a/system/vdagent/src/main.rs b/system/vdagent/src/main.rs index 2cc4b472c8c..606aec45d97 100644 --- a/system/vdagent/src/main.rs +++ b/system/vdagent/src/main.rs @@ -9,7 +9,8 @@ //! //! **The handshake is virtio 1.2 §5.3.6.2's**: `DEVICE_READY`, the device's //! `DEVICE_ADD` of [`PORT_ID`] answered with `PORT_READY`, and the agent's -//! port opened once the host's `PORT_OPEN(1)` has come. The host's +//! port opened once the host's `PORT_OPEN(1)` has come. One process runs per +//! virtio-serial function the kernel does not drive, holding its claim. The host's //! `PORT_OPEN(0)` is answered with the agent's `PORT_OPEN(0)`, and its //! `PORT_OPEN(1)` with `PORT_OPEN(1)`: QEMU 11.1.1 then starts a new session //! with its capabilities, measured. Any other port the device adds, a console @@ -30,6 +31,7 @@ use toyos::poller::{Poller, READABLE}; use toyos::say; use toyos::PciDev; use toyos_abi::syscall::PciId; +use toyos_manifest::device::PciMatch; use toyos_vdagent::control::{Control, Event}; use toyos_vdagent::stream::{Decoder, Message}; use toyos_vdagent::PORT_ID; @@ -45,8 +47,10 @@ use channel::Channel; /// compositor alike. const _: () = assert!(toyos_vdagent::MAX_TEXT == window::MAX_CLIPBOARD_BYTES); -/// Virtio 1.2 §5.3.1: the console device is `0x1040 + 3`. -const VIRTIO_SERIAL: PciId = PciId { vendor: 0x1af4, device: 0x1043 }; +/// A communication controller of class `07:80:00`, which virtio 1.2 §5.3.1 +/// numbers `0x1040 + 3`: the matcher its row drives, and the label its claim +/// is endowed under. +const VIRTIO_SERIAL: PciMatch = PciMatch::new([0x07, 0x80, 0x00], Some(PciId { vendor: 0x1af4, device: 0x1043 })); /// How long a transmit may stay with the device. Policy, far past any time /// QEMU 11.1.1 took, measured: it took every element at once. @@ -157,10 +161,8 @@ impl Agent { } fn main() { - let Some(claim) = endow::pci_function::(VIRTIO_SERIAL) else { - say!("vdagent: no virtio-serial function was handed to this program, exiting"); - return; - }; + let claim: PciDev = endow::claim(&VIRTIO_SERIAL.to_string()) + .expect("vdagent: its row drives a function, and it is started holding the function's claim"); let watcher = Watcher::connect(toyos_manifest::clipboard::PORT) .unwrap_or_else(|why| panic!("vdagent: the compositor's clipboard: {why}")); let mut channel = Channel::open(claim).unwrap_or_else(|why| panic!("vdagent: the function is not driven: {why}")); diff --git a/tests/common/iommu.rs b/tests/common/iommu.rs index 57a2dd6c83c..77dff741106 100644 --- a/tests/common/iommu.rs +++ b/tests/common/iommu.rs @@ -183,9 +183,9 @@ fn not_handed_over(at: &str, why: &str) -> String { fn no_unit_is_no_claim(log: &Serial) -> Result<(), String> { refused_claim(log, NETSTACK_CLAIMS, NOT_REMAPPED, &[NVME_AT, SOUND_AT])?; log.must_say(¬_handed_over(SOUND_AT, NOT_REMAPPED))?; - log.must_say("supervisor: soundserver: pci:1af4:1059 is on this machine and could not be handed over")?; + log.must_say("supervisor: soundserver: pci:04:01:00/1af4:1059 is on this machine and could not be handed over")?; log.must_say(¬_handed_over(NVME_AT, NOT_REMAPPED))?; - log.must_say("supervisor: diskserver: pci:1b36:0010 is on this machine and could not be handed over")?; + log.must_say("supervisor: diskserver: pci:01:08:02/1b36:0010 is on this machine and could not be handed over")?; // And this machine handed *nothing* over, which is more than the claim's // own refusal says: with no unit there is no function any process could be // given an address space for. @@ -317,7 +317,7 @@ fn refused_claim(log: &Serial, claims: &str, why: &str, beside: &[&str]) -> Resu // config's own spelling, and **with this refusal's own word**: the machine // has the function, so "no such device on this machine" would be false. log.must_say(&format!( - "supervisor: netstack: pci:{claims} is on this machine and could not be handed over" + "supervisor: netstack: pci:02:00:00/{claims} is on this machine and could not be handed over" ))?; Ok(()) } diff --git a/tests/common/qemu.rs b/tests/common/qemu.rs index 4c020f44dd6..41355514685 100644 --- a/tests/common/qemu.rs +++ b/tests/common/qemu.rs @@ -691,6 +691,10 @@ pub enum Profile { /// the T14's two are, because a claim never maps the BAR that holds an /// MSI-X table and QEMU puts this one's in its registers' BAR. HeadlessUsbSpare, + /// [`Profile::Headless`] with two of QEMU's `edu`, which the kernel drives + /// neither of: two identical functions, each a `drives` row's to start a + /// process on. + HeadlessTwoEdu, /// M1 metal-sim: GOP, NVMe, xHCI with the boot stick on it, i8042 from /// q35, and nothing else -- no virtio device and no USB HID. This is the /// machine shape that gets flashed, so it is the one the input tests run @@ -770,6 +774,7 @@ impl Profile { | Self::HeadlessNoUsb | Self::HeadlessVirtioGpu | Self::HeadlessUsbSpare + | Self::HeadlessTwoEdu | Self::Metal | Self::MetalNoX2apic | Self::MetalAmdVi @@ -942,6 +947,9 @@ enum Probes { /// Two of QEMU's `iommu-testdev`, the only DMA a guest of this suite can /// aim at an address of its choosing, behind the machine's unit. Testdevs, + /// Two of QEMU's `edu`, behind the machine's unit: identical functions a + /// claim tells apart only by their arrivals. + Edus, /// On `virt`, its SMMUv3 with the ITS behind it /// ([`Arch::irqchip`]), one `edu`, whose message is the only one a guest /// of this suite can raise on demand, and an `e1000e` behind a root port @@ -1049,6 +1057,7 @@ impl Profile { Self::HeadlessNoIommu => Shape { iommu: None, ..Self::Headless.shape() }, Self::HeadlessNoUsb => Shape { xhci: &[], usb: &[], ..Self::Headless.shape() }, Self::HeadlessVirtioGpu => Shape { virtio_gpu: true, ..Self::Headless.shape() }, + Self::HeadlessTwoEdu => Shape { probes: Probes::Edus, ..Self::Headless.shape() }, Self::HeadlessUsbSpare => Shape { xhci: &[XHCI_DEFAULT, XHCI_SPARE], usb: &[ @@ -2147,8 +2156,8 @@ fn qemu_command( machine.extend(arch.irqchip(accel, remapped)); match arch { Arch::X86_64 => assert!( - shape.probes == Probes::Absent || (shape.probes == Probes::Testdevs && shape.iommu.is_some()), - "a q35 has no SMMUv3, and an iommu-testdev on one writes through no unit" + shape.probes == Probes::Absent || (matches!(shape.probes, Probes::Testdevs | Probes::Edus) && shape.iommu.is_some()), + "a q35 has no SMMUv3, and an iommu-testdev or edu on one is behind no unit" ), Arch::Aarch64 => { // A unit a profile declares is VT-d or AMD-Vi, which `virt` has none of. @@ -2304,6 +2313,10 @@ fn qemu_command( qemu.arg("-device").arg("iommu-testdev"); qemu.arg("-device").arg("iommu-testdev"); } + if shape.probes == Probes::Edus { + qemu.arg("-device").arg("edu"); + qemu.arg("-device").arg("edu"); + } if shape.probes == Probes::Its { qemu.arg("-device").arg("edu"); qemu.arg("-device").arg("pxb-pcie,id=bypass,bus_nr=8,bypass_iommu=on"); diff --git a/tests/guest/pciholder/Cargo.toml b/tests/guest/pciholder/Cargo.toml new file mode 100644 index 00000000000..da6675a8cd5 --- /dev/null +++ b/tests/guest/pciholder/Cargo.toml @@ -0,0 +1,9 @@ +[package] +name = "pciholder" +description = "Test-only: a drives row's process, which says which PCI function its claim holds and keeps holding it until the boot ends." +version = "0.1.0" +edition = "2021" +license = "MIT OR Apache-2.0" + +[dependencies] +toyos = { path = "../../../sdk/toyos" } diff --git a/tests/guest/pciholder/src/main.rs b/tests/guest/pciholder/src/main.rs new file mode 100644 index 00000000000..42224e9da7b --- /dev/null +++ b/tests/guest/pciholder/src/main.rs @@ -0,0 +1,23 @@ +//! A `drives` row's process for a test boot: it says which function the claim +//! it was started holding drives, by the address the kernel hands over, and +//! holds it until the boot ends. + +use toyos::endow::{Endowments, DEV_PREFIX}; +use toyos::PciDev; + +/// What it says once it holds its function, followed by the function's +/// `bus:dev.func`. +const HOLDING: &str = "pciholder: holding PCI "; + +fn main() { + let endowed = Endowments::get(); + let mut labels = endowed.labels().filter(|label| label.strip_prefix(DEV_PREFIX).is_some_and(|name| name.starts_with("pci:"))); + let label = labels.next().map(str::to_string).expect("pciholder: started with no `dev:pci:` claim"); + assert!(labels.next().is_none(), "pciholder: started with two `dev:pci:` claims"); + let claim: PciDev = endowed.take(&label).expect("pciholder: the claim its label names"); + let info = claim.describe().expect("pciholder: its claim's description"); + println!("{HOLDING}{:02x}:{:02x}.{}", info.bus, info.dev, info.func); + loop { + std::thread::park(); + } +} diff --git a/tests/guest/rust-tests/src/bin/bar_map_again.rs b/tests/guest/rust-tests/src/bin/bar_map_again.rs index 3e540482f13..7c461b543ce 100644 --- a/tests/guest/rust-tests/src/bin/bar_map_again.rs +++ b/tests/guest/rust-tests/src/bin/bar_map_again.rs @@ -9,6 +9,9 @@ //! The function is QEMU's virtio NIC, which no program of `tests/images/testcases.toml` //! holds. +#[path = "../function.rs"] +mod function; + use toyos::endow::Endowments; use toyos::shm::SharedMemory; use toyos::syscap::SysCap; @@ -46,7 +49,7 @@ fn main() { let cap: SysCap = Endowments::get() .take(SYSCAP_LABEL) .expect("the test estate is endowed a device-minting capability"); - let nic: PciDev = cap.claim_pci(NIC).expect("bar_map_again: the virtio NIC's claim"); + let nic: PciDev = function::claim(&cap, NIC).expect("bar_map_again: the virtio NIC's claim"); let info = nic.describe().expect("bar_map_again: the claim's description"); let bar = info .bar_bytes diff --git a/tests/guest/rust-tests/src/bin/claim_bar_fault.rs b/tests/guest/rust-tests/src/bin/claim_bar_fault.rs index 98a5a563326..f954366594a 100644 --- a/tests/guest/rust-tests/src/bin/claim_bar_fault.rs +++ b/tests/guest/rust-tests/src/bin/claim_bar_fault.rs @@ -11,6 +11,9 @@ //! harness reds on either marker anywhere on the console, which is where a //! report that read through the BAR would put it. +#[path = "../function.rs"] +mod function; + #[path = "../arch/stack.rs"] mod stack; @@ -65,7 +68,7 @@ fn endowed() -> SysCap { } fn claim(cap: &SysCap) -> PciDev { - cap.claim_pci(EDU).unwrap_or_else(|why| panic!("claim_bar_fault: the claim on edu was refused: {why:?}")) + function::claim(&cap, EDU).unwrap_or_else(|why| panic!("claim_bar_fault: the claim on edu was refused: {why:?}")) } /// BAR 0 mapped, and its registers. diff --git a/tests/guest/rust-tests/src/bin/claim_lpi.rs b/tests/guest/rust-tests/src/bin/claim_lpi.rs index 0c3a305b14d..7e878f34a61 100644 --- a/tests/guest/rust-tests/src/bin/claim_lpi.rs +++ b/tests/guest/rust-tests/src/bin/claim_lpi.rs @@ -5,6 +5,9 @@ //! The function masters the bus only once it has a grant, and QEMU sends an //! MSI as a write the function makes, so one grant comes first. +#[path = "../function.rs"] +mod function; + use std::time::Duration; use toyos::endow::{Endowments, SYSCAP_LABEL}; @@ -34,7 +37,7 @@ const BOUND: Duration = Duration::from_secs(10); fn main() { let cap: SysCap = Endowments::get().take(SYSCAP_LABEL).expect("test-runner endows a device-minting capability"); - let dev: PciDev = cap.claim_pci(EDU).unwrap_or_else(|why| panic!("claim_lpi: the claim on edu was refused: {why:?}")); + let dev: PciDev = function::claim(&cap, EDU).unwrap_or_else(|why| panic!("claim_lpi: the claim on edu was refused: {why:?}")); let info = dev.describe().expect("claim_lpi: the claim's description"); let bar = dev.map_bar(0, info.bar_bytes[0]).expect("claim_lpi: BAR 0"); // SAFETY: the mapping is the BAR's length and lives as long as `bar`. diff --git a/tests/guest/rust-tests/src/bin/claim_unrouted.rs b/tests/guest/rust-tests/src/bin/claim_unrouted.rs index 768af6b1f2c..bdf8b6ade3e 100644 --- a/tests/guest/rust-tests/src/bin/claim_unrouted.rs +++ b/tests/guest/rust-tests/src/bin/claim_unrouted.rs @@ -3,6 +3,9 @@ //! and the kernel that refused it goes on: the next claim, of `edu`, is //! handed over. +#[path = "../function.rs"] +mod function; + use toyos::endow::{Endowments, SYSCAP_LABEL}; use toyos::syscap::SysCap; use toyos::PciDev; @@ -15,8 +18,8 @@ const EDU: PciId = PciId { vendor: 0x1234, device: 0x11e8 }; fn main() { let cap: SysCap = Endowments::get().take(SYSCAP_LABEL).expect("test-runner endows a device-minting capability"); - let refused = cap.claim_pci::(E1000E).map(|_| ()); + let refused = function::claim::(&cap, E1000E).map(|_| ()); assert_eq!(refused, Err(SyscallError::NotSupported), "claim_unrouted: the claim on e1000e was not refused as unusable"); - let _edu: PciDev = cap.claim_pci(EDU).unwrap_or_else(|why| panic!("claim_unrouted: edu, claimed after, was refused: {why:?}")); + let _edu: PciDev = function::claim(&cap, EDU).unwrap_or_else(|why| panic!("claim_unrouted: edu, claimed after, was refused: {why:?}")); println!("claim_unrouted: e1000e, routed past the SMMUv3, was refused, and edu was claimed after"); } diff --git a/tests/guest/rust-tests/src/bin/drives_judged.rs b/tests/guest/rust-tests/src/bin/drives_judged.rs new file mode 100644 index 00000000000..8255ac4a5a0 --- /dev/null +++ b/tests/guest/rust-tests/src/bin/drives_judged.rs @@ -0,0 +1,43 @@ +//! A `drives` row's processes as the inventory sees them: each of QEMU's two +//! `edu` functions claimed, each by a `pciholder` of its own, still holding it +//! when this runs. + +use toyos::endow::{Endowments, SYSCAP_LABEL}; +use toyos::syscap::SysCap; +use toyos_abi::inventory::{Claim, Claimed, Driven, Pci, RawRecord, Record}; + +const EDU: (u16, u16) = (0x1234, 0x11e8); +const HOLDER: &str = "pciholder"; + +fn main() { + let cap: SysCap = Endowments::get().take(SYSCAP_LABEL).expect("drives_judged: test-runner endows an inventory-reading capability"); + let records: Vec = + cap.records(|n| vec![RawRecord::EMPTY; n]).unwrap_or_else(|why| panic!("drives_judged: the inventory: {why:?}")); + let edus: Vec<&Pci> = records + .iter() + .filter_map(|record| match record { + Record::Pci(pci) if (pci.vendor, pci.device) == EDU => Some(pci), + _ => None, + }) + .collect(); + assert_eq!(edus.len(), 2, "drives_judged: this machine has {} edu functions, not two", edus.len()); + let mut held = Vec::new(); + for edu in edus { + let at = format!("{:02x}:{:02x}.{}", edu.at.bus, edu.at.dev, edu.at.func); + assert_eq!(edu.driven, Driven::Claimed, "drives_judged: PCI {at} is not held"); + let holder = records + .iter() + .find_map(|record| match record { + Record::Claim(Claim { on: Claimed::Pci(on), holder }) if *on == edu.at => Some(*holder), + _ => None, + }) + .unwrap_or_else(|| panic!("drives_judged: no claim record names PCI {at}")); + assert_eq!(holder.name(), Some(HOLDER), "drives_judged: PCI {at} is held by {:?}", holder.name()); + held.push((at, holder.pid)); + } + assert_ne!(held[0].1, held[1].1, "drives_judged: one process holds both functions: {held:?}"); + println!( + "drives_judged: PCI {} held by {HOLDER} pid {}, PCI {} by {HOLDER} pid {}", + held[0].0, held[0].1, held[1].0, held[1].1 + ); +} diff --git a/tests/guest/rust-tests/src/bin/end_returns_claims.rs b/tests/guest/rust-tests/src/bin/end_returns_claims.rs index 09c23f1e13a..6356bd18fd8 100644 --- a/tests/guest/rust-tests/src/bin/end_returns_claims.rs +++ b/tests/guest/rust-tests/src/bin/end_returns_claims.rs @@ -7,6 +7,9 @@ //! - no argument: the rounds. //! - `hold`: the child, which holds the claim it was endowed until it is killed. +#[path = "../function.rs"] +mod function; + use std::io::{BufRead, BufReader}; use std::os::toyos::process::CommandExt; use std::process::{Command, Stdio}; @@ -15,7 +18,7 @@ use std::time::{Duration, Instant}; use toyos::endow::{Endowments, SYSCAP_LABEL}; use toyos::syscap::SysCap; use toyos::PciDev; -use toyos_abi::syscall::{DeviceRequest, PciId, SyscallError}; +use toyos_abi::syscall::{PciId, SyscallError}; const SELF_PATH: &str = "/system/bin/test_rs_end_returns_claims"; @@ -40,15 +43,14 @@ fn main() { } fn spelled(id: PciId) -> String { - let mut buf = [0u8; DeviceRequest::MAX_NAME]; - DeviceRequest::Pci(id).write_name(&mut buf).to_string() + format!("pci:{:04x}:{:04x}", id.vendor, id.device) } /// The one spare this machine has, claimed. fn claim(cap: &SysCap) -> (PciId, PciDev) { let mut found = None; for id in SPARES { - match cap.claim_pci::(id) { + match function::claim::(&cap, id) { Err(SyscallError::NotFound) => {} Ok(dev) => { assert!(found.is_none(), "end_returns_claims: two spares on one machine"); @@ -84,7 +86,7 @@ fn rounds() { let status = child.wait().expect("end_returns_claims: wait for the holder"); assert!(!status.success(), "end_returns_claims: a killed holder ended {status:?}"); let waited = Instant::now(); - dev = cap.claim_pci::(id).unwrap_or_else(|why| { + dev = function::claim::(&cap, id).unwrap_or_else(|why| { panic!("end_returns_claims: round {round}: the claim after its holder's end was refused: {why:?}") }); ended += waited - killed; diff --git a/tests/guest/rust-tests/src/bin/i219_msi.rs b/tests/guest/rust-tests/src/bin/i219_msi.rs index 252afc4af79..b3cab4e94f9 100644 --- a/tests/guest/rust-tests/src/bin/i219_msi.rs +++ b/tests/guest/rust-tests/src/bin/i219_msi.rs @@ -8,6 +8,9 @@ //! The PCH datasheet (631120 §8.1) and the 82574's (§10.2.4) are what each //! reading is held to; no other machine has this function. +#[path = "../function.rs"] +mod function; + use std::rc::Rc; use std::time::Duration; @@ -237,7 +240,7 @@ fn send(driver: &mut Driver, frames: &Window, bytes: &[u8]) { fn main() { let cap: SysCap = Endowments::get().take(SYSCAP_LABEL).expect("the test estate is endowed a device-minting capability"); - let dev = Rc::new(cap.claim_pci::(I219_ID).unwrap_or_else(|why| fail(&format!("the claim was refused: {why:?}")))); + let dev = Rc::new(function::claim::(&cap, I219_ID).unwrap_or_else(|why| fail(&format!("the claim was refused: {why:?}")))); let info = dev.describe().unwrap_or_else(|why| fail(&format!("the description was refused: {why:?}"))); let (index, bytes) = info .bar_bytes diff --git a/tests/guest/rust-tests/src/bin/lent_device_memory_refused.rs b/tests/guest/rust-tests/src/bin/lent_device_memory_refused.rs index 52e5bf68293..5e41fe1c2ab 100644 --- a/tests/guest/rust-tests/src/bin/lent_device_memory_refused.rs +++ b/tests/guest/rust-tests/src/bin/lent_device_memory_refused.rs @@ -5,6 +5,9 @@ //! The device is QEMU's virtio NIC, which no program of //! `tests/images/testcases.toml` holds. +#[path = "../function.rs"] +mod function; + use toyos::endow::Endowments; use toyos::fs::WINDOW_BYTES; use toyos::shm::{Lent, SharedMemory}; @@ -16,7 +19,7 @@ const NIC: PciId = PciId { vendor: 0x1af4, device: 0x1041 }; fn main() { let cap: SysCap = Endowments::get().take(SYSCAP_LABEL).expect("lent_device_memory_refused: a device-minting capability"); - let nic: PciDev = cap.claim_pci(NIC).expect("lent_device_memory_refused: the virtio NIC's claim"); + let nic: PciDev = function::claim(&cap, NIC).expect("lent_device_memory_refused: the virtio NIC's claim"); let info = nic.describe().expect("lent_device_memory_refused: the claim's description"); let bar = info.bar_bytes.iter().position(|&bytes| bytes != 0).expect("lent_device_memory_refused: a memory BAR"); let bar = syscall::device_bar_map(nic.as_handle(), bar as u32).expect("lent_device_memory_refused: the BAR's region"); diff --git a/tests/guest/rust-tests/src/bin/pci_reclaim.rs b/tests/guest/rust-tests/src/bin/pci_reclaim.rs index db180f18fba..b26b8394c53 100644 --- a/tests/guest/rust-tests/src/bin/pci_reclaim.rs +++ b/tests/guest/rust-tests/src/bin/pci_reclaim.rs @@ -2,6 +2,9 @@ //! is whether both claims were handed over. What each claim spent at the unit //! is the kernel's to say, and the metal rows read it there. +#[path = "../function.rs"] +mod function; + use toyos::endow::Endowments; use toyos::syscap::SysCap; use toyos::PciDev; @@ -14,7 +17,7 @@ fn main() { .take(SYSCAP_LABEL) .expect("the test estate is endowed a device-minting capability"); for claim in 1..=2 { - match cap.claim_pci::(I219) { + match function::claim::(&cap, I219) { // Dropped at once: the release is what the second claim follows. Ok(function) => drop(function), Err(why) => { diff --git a/tests/guest/rust-tests/src/bin/usbd_spare.rs b/tests/guest/rust-tests/src/bin/usbd_spare.rs index d43b2739468..8ae4723881e 100644 --- a/tests/guest/rust-tests/src/bin/usbd_spare.rs +++ b/tests/guest/rust-tests/src/bin/usbd_spare.rs @@ -12,6 +12,9 @@ //! and the claim refusing its interrupt read from then on is this job's. //! - `refused`: the claim, on a kernel that drives the function, refused. +#[path = "../function.rs"] +mod function; + use std::collections::BTreeMap; use std::io::{BufRead, BufReader}; use std::os::toyos::process::CommandExt; @@ -25,7 +28,7 @@ use toyos::poller::{Poller, READABLE}; use toyos::syscap::SysCap; use toyos::volatile::Window; use toyos::PciDev; -use toyos_abi::syscall::{DeviceRequest, PciId, SyscallError}; +use toyos_abi::syscall::{PciId, SyscallError}; use toyos_inspect::{Value, MSG_INSPECT, MSG_SNAPSHOT, USB}; const USBD: &str = "/system/bin/usbd"; @@ -61,7 +64,7 @@ fn main() { fn claim(cap: &SysCap) -> (PciId, Result) { let mut found = None; for id in SPARES { - match cap.claim_pci::(id) { + match function::claim::(&cap, id) { Err(SyscallError::NotFound) => {} answered => { assert!(found.is_none(), "usbd_spare: two spares on one machine"); @@ -73,8 +76,7 @@ fn claim(cap: &SysCap) -> (PciId, Result) { } fn spelled(id: PciId) -> String { - let mut buf = [0u8; DeviceRequest::MAX_NAME]; - DeviceRequest::Pci(id).write_name(&mut buf).to_string() + format!("pci:{:04x}:{:04x}", id.vendor, id.device) } /// usbd, started on `dev` and serving its port to this job, and the lines it diff --git a/tests/guest/rust-tests/src/function.rs b/tests/guest/rust-tests/src/function.rs new file mode 100644 index 00000000000..9bd17857a78 --- /dev/null +++ b/tests/guest/rust-tests/src/function.rs @@ -0,0 +1,27 @@ +//! The PCI function a job claims, found in the inventory by what identifies +//! its card: a claim names a function by its arrival. + +use toyos::endow::FromHandle; +use toyos::syscap::SysCap; +use toyos_abi::inventory::{Pci, RawRecord, Record}; +use toyos_abi::syscall::{PciId, SyscallError}; + +/// The one function of card `id` this machine has, whoever drives it, or +/// `None`. Two of one card is a machine the job was not written for. +pub fn of(cap: &SysCap, id: PciId) -> Option { + let records: Vec = + cap.records(|n| vec![RawRecord::EMPTY; n]).unwrap_or_else(|why| panic!("the job's capability reads no inventory: {why:?}")); + let mut found = records.into_iter().filter_map(|record| match record { + Record::Pci(pci) if pci.vendor == id.vendor && pci.device == id.device => Some(pci), + _ => None, + }); + let one = found.next()?; + assert!(found.next().is_none(), "this machine has two functions of {:04x}:{:04x}", id.vendor, id.device); + Some(one) +} + +/// The claim on the one function of card `id`: `NotFound` where the machine +/// has none. +pub fn claim(cap: &SysCap, id: PciId) -> Result { + of(cap, id).map_or(Err(SyscallError::NotFound), |pci| cap.claim_function(pci.arrival)) +} diff --git a/tests/images/acpicase.toml b/tests/images/acpicase.toml index b2916230bd3..b7fdf1134bf 100644 --- a/tests/images/acpicase.toml +++ b/tests/images/acpicase.toml @@ -36,7 +36,7 @@ args = ["test_rs_acpi_mediated", "reboot"] service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/tests/images/clipboardcase.toml b/tests/images/clipboardcase.toml index d4db604830d..3976ac1c957 100644 --- a/tests/images/clipboardcase.toml +++ b/tests/images/clipboardcase.toml @@ -8,7 +8,7 @@ assets = ["assets"] [boot] -start = ["logkeeper", "compositor", "vdagent", "test-runner"] +start = ["logkeeper", "compositor", "test-runner"] # `every_boot_config_runs_logkeeper` is what refuses a boot config without it. [programs.logkeeper] @@ -20,8 +20,10 @@ service = true serves = ["compositor", "prompt", "clipboard", "input"] devices = ["framebuffer", "keyboard", "mouse"] +# One process per virtio-serial function the kernel does not drive: the +# clipboard's channel, beside the kernel's console. [programs.vdagent] -devices = ["pci:1af4:1043"] +drives = ["pci:07:80:00/1af4:1043"] receives = ["clipboard"] [programs.test-runner] diff --git a/tests/images/drivescase.toml b/tests/images/drivescase.toml new file mode 100644 index 00000000000..e8ed7433d85 --- /dev/null +++ b/tests/images/drivescase.toml @@ -0,0 +1,21 @@ +# The boot `a_drives_row_holds_one_process_per_function` reads, on +# `Profile::HeadlessTwoEdu`: one `pciholder` per `edu` function, each holding +# its own until the job that judges them has run. + +[boot] +start = ["logkeeper", "test-runner"] + +# `every_boot_config_runs_logkeeper` is what refuses a boot config without it. +[programs.logkeeper] +service = true +syscap = ["logread"] + +# Started on each function the kernel does not drive, never by `[boot] start`. +[programs.pciholder] +drives = ["pci:00:ff:00/1234:11e8"] + +# `inventory` because the job reads who holds each function, and `dup` +# because test-runner hands a job its capability as a duplicate and hands +# none without it. +[programs.test-runner] +syscap = ["inventory", "dup"] diff --git a/tests/images/jobcase.toml b/tests/images/jobcase.toml index 7c20226ea7c..252c685e28f 100644 --- a/tests/images/jobcase.toml +++ b/tests/images/jobcase.toml @@ -27,7 +27,7 @@ args = ["reboot"] service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/tests/images/latencycase.toml b/tests/images/latencycase.toml index 30f91e1a01c..cb4496bb0f2 100644 --- a/tests/images/latencycase.toml +++ b/tests/images/latencycase.toml @@ -35,7 +35,7 @@ syscap = ["rt", "dup", "counters"] service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/tests/images/metalcase.toml b/tests/images/metalcase.toml index 68dd34b70f7..f20c0c24aee 100644 --- a/tests/images/metalcase.toml +++ b/tests/images/metalcase.toml @@ -33,7 +33,7 @@ devices = ["framebuffer", "keyboard", "mouse"] [programs.soundserver] service = true serves = ["soundserver", "volume"] -devices = ["hda-audio", "pci:1af4:1059"] +devices = ["hda-audio", "pci:04:01:00/1af4:1059"] syscap = ["rt"] # netstack holds the NIC's PCI function and drives it: the virtqueues, the register @@ -41,7 +41,7 @@ syscap = ["rt"] [programs.netstack] service = true serves = ["netstack"] -devices = ["pci:1af4:1041"] +devices = ["pci:02:00:00/1af4:1041"] [programs.sshserver] service = true @@ -71,7 +71,7 @@ syscap = ["logread"] service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/tests/images/metaldevicecase.toml b/tests/images/metaldevicecase.toml index a31413e96f5..594c145fe4b 100644 --- a/tests/images/metaldevicecase.toml +++ b/tests/images/metaldevicecase.toml @@ -60,7 +60,7 @@ syscap = ["device", "dup"] service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/tests/images/netcase.toml b/tests/images/netcase.toml index a369635eb0b..556f9a2a698 100644 --- a/tests/images/netcase.toml +++ b/tests/images/netcase.toml @@ -20,17 +20,17 @@ syscap = ["logread"] # netstack holds the NIC's PCI function and drives it: the virtqueues, the register # window and the interrupt are its own, and the kernel keeps only the claim. -# Named by vendor and device rather than by slot, so one row finds the card +# Named by class, vendor and device rather than by slot, so one row finds the card # wherever firmware put it. [programs.netstack] service = true serves = ["netstack"] -devices = ["pci:1af4:1041"] +devices = ["pci:02:00:00/1af4:1041"] [programs.soundserver] service = true serves = ["soundserver", "volume"] -devices = ["pci:1af4:1059"] +devices = ["pci:04:01:00/1af4:1059"] syscap = ["rt"] [programs.test-runner] @@ -42,7 +42,7 @@ syscap = ["rt"] service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/tests/images/proctreecase.toml b/tests/images/proctreecase.toml index e878af7d576..d5cf2de0250 100644 --- a/tests/images/proctreecase.toml +++ b/tests/images/proctreecase.toml @@ -55,7 +55,7 @@ receives = ["grants"] service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/tests/images/testcases.toml b/tests/images/testcases.toml index 3febe42a233..75cc65f21d9 100644 --- a/tests/images/testcases.toml +++ b/tests/images/testcases.toml @@ -26,7 +26,7 @@ receives = ["power"] [programs.soundserver] service = true serves = ["soundserver", "volume"] -devices = ["hda-audio", "pci:1af4:1059"] +devices = ["hda-audio", "pci:04:01:00/1af4:1059"] syscap = ["rt"] # The test estate's authority. @@ -52,9 +52,11 @@ syscap = ["rt"] # steps it. # `log` because `counters_metal` measures its idle second only once the log # holds its own line, read back off logkeeper. +# `inventory` because a claim names a PCI function by its arrival, which the +# jobs that claim one read off the inventory. [programs.test-runner] receives = ["soundserver", "volume", "power", "log"] -syscap = ["device", "dup", "logread", "power", "roster", "counters", "trace"] +syscap = ["device", "dup", "logread", "power", "roster", "counters", "trace", "inventory"] [programs.toybox] receives = ["soundserver"] @@ -91,7 +93,7 @@ receives = ["soundserver"] service = true restart = true serves = ["block"] -devices = ["pci:1b36:0010"] +devices = ["pci:01:08:02/1b36:0010"] # The file servers: DATA, the log and the running slot's volume, one process # each, serving every program the directories of its role. Started again when diff --git a/tests/images/virtclaimcase.toml b/tests/images/virtclaimcase.toml index 62d54be0318..518a31548f1 100644 --- a/tests/images/virtclaimcase.toml +++ b/tests/images/virtclaimcase.toml @@ -10,9 +10,11 @@ start = ["logkeeper", "test-runner"] service = true syscap = ["logread"] -# `device` because the job mints the claim, `dup` because test-runner hands a -# job its capability as a duplicate and hands none without it. +# `device` because the job mints the claim, `inventory` because it names the +# function by the arrival the inventory gives it, and `dup` because +# test-runner hands a job its capability as a duplicate and hands none +# without it. [programs.test-runner] receives = ["power"] args = ["--bound-ms=240000", "test_rs_claim_lpi", "test_rs_claim_unrouted", "test_rs_claim_bar_fault"] -syscap = ["device", "dup"] +syscap = ["device", "inventory", "dup"] diff --git a/tests/toyos.rs b/tests/toyos.rs index 86f930e33cd..5745cc705c9 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -203,6 +203,9 @@ const RUST_SKIP: &[&str] = &[ // The same controller, handed to a holder that is killed and claimed // again at once, in the same boots as `usbd_spare`. "end_returns_claims", + // It reads two `edu` functions each held by a `drives` row's process, + // which only `a_drives_row_holds_one_process_per_function`'s boot has. + "drives_judged", ]; /// The shared boot's last members, in this order: each fills a bound of its @@ -347,6 +350,13 @@ fn gop_edid_scale(loader: &str, log: &str, kernel: &str) -> Result<(), String> { /// The tests whose machine shape *is* the test, each on a boot of its own. /// `run_machine_test` dispatches them. const MACHINE_TESTS: &[&str] = &[ + // A `drives` row's one process per function, on QEMU x86-64's q35 with + // two `edu` (`Profile::HeadlessTwoEdu`): which functions get one is + // `toyos-supervisor`'s, host-tested, and this is the supervisor + // performing it — two claims by arrival on identical functions, two + // spawns, each process holding its own — which no host build has. No + // bench has two identical functions the kernel leaves to a claim. + "a_drives_row_holds_one_process_per_function", // Whether QEMU's virtio functions negotiated `VIRTIO_F_ACCESS_PLATFORM` // behind its emulated VT-d unit and without one: no shipped machine has a // virtio function, so only a QEMU machine can be asked. @@ -3725,6 +3735,7 @@ fn run_machine_test(name: &str, test_config: &Path) -> Result<(), String> { "nvme_disk_boots" => nvme_disk_boots(test_config), "hosted_clang_hello" => common::hostedclang::hosted_clang_hello(), "usbd_drives_the_spare" => usbd_drives_the_spare(test_config), + "a_drives_row_holds_one_process_per_function" => a_drives_row_holds_one_process_per_function(), "usb_keyboard_rollover" => usb_keyboard_rollover(test_config), "vector_state_survives_a_switch" => vector_state_survives_a_switch(test_config), "the_opening_exchange_with_qemus_agent_completes" => the_opening_exchange_with_qemus_agent_completes(), @@ -3878,6 +3889,43 @@ fn usbd_drives_the_spare(test_config: &Path) -> Result<(), String> { Ok(()) } +/// A `drives` row's processes on a q35 with two `edu`, which the kernel drives +/// neither of: the kernel hands each function over once, two `pciholder`s +/// each say they hold one of the two, and the job reads both still held, +/// each by a process of its own. +fn a_drives_row_holds_one_process_per_function() -> Result<(), String> { + const JOB: &str = "drives_judged"; + const HOLDING: &str = "pciholder: holding PCI "; + const HANDED: &str = " [1234:11e8] handed over, "; + let options = BootOptions { profile: qemu::Profile::HeadlessTwoEdu, ..Default::default() }; + let edus = qemu::profile_argv(&options).iter().filter(|arg| *arg == "edu").count(); + if edus != 2 { + return Err(format!("the machine was to have two edu functions, and QEMU is given {edus}")); + } + let bin = qemu::build_toyos_bin(qemu::SUITE_ARCH, &compile::repo_root().join("tests/guest/rust-tests"), JOB); + let config = compile::repo_root().join("tests/images/drivescase.toml"); + let mut qemu = QemuInstance::boot_with_options(&config, &[], &[(JOB.to_string(), bin)], options); + let mut console = format!("{}\n", qemu.boot_log()); + await_guest(&mut qemu, &mut console, "both holders' lines", |t| t.matches(HOLDING).count() >= 2) + .map_err(|e| format!("{e}\n{console}"))?; + let said = serial::Serial::named("the drives boot", console.clone()); + said.must_be_clean()?; + let handed: BTreeSet<&str> = + console.lines().filter_map(|l| l.split_once("pcidev: PCI ")?.1.split_once(HANDED).map(|(at, _)| at)).collect(); + let holding: BTreeSet<&str> = + console.lines().filter_map(|l| l.split_once(HOLDING)?.1.split_whitespace().next()).collect(); + if handed.len() != 2 || holding != handed { + return Err(format!("the kernel handed over {handed:?} and the holders say they hold {holding:?}:\n{console}")); + } + let judged = job_window(&mut qemu, "test_rs_drives_judged")?; + let line = judged + .lines() + .find(|l| l.contains("drives_judged: PCI ")) + .ok_or_else(|| format!("the job never said who holds the functions:\n{judged}"))?; + eprintln!(" [drives] {}", line.trim()); + Ok(()) +} + /// Every AMD-Vi unit translating, every xHCI controller the kernel drives /// moved to a domain of its own, the boot stick bound through one, and no /// DMA refused. From 37cd4e1a786e936538eaed8aebceba0f7bbb1a33 Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 14 Oct 2026 17:40:07 +0200 Subject: [PATCH 2/6] manifest: the refusal test names its row tuple, which clippy calls too complex Co-Authored-By: Claude Opus 5.5 --- lib/manifest/src/lib.rs | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/manifest/src/lib.rs b/lib/manifest/src/lib.rs index 15ef6337701..ebf1978a28f 100644 --- a/lib/manifest/src/lib.rs +++ b/lib/manifest/src/lib.rs @@ -916,7 +916,10 @@ mod tests { ); } - fn with_rows(rows: &[(&str, &[&str], &[&str])]) -> Manifest { + /// A row's name, its `devices` and its `drives`. + type Row<'a> = (&'a str, &'a [&'a str], &'a [&'a str]); + + fn with_rows(rows: &[Row<'_>]) -> Manifest { Manifest { programs: rows .iter() @@ -944,7 +947,7 @@ mod tests { /// and a `drives` matcher alike, and a fixed name written twice. #[test] fn two_entries_one_device_can_answer_are_refused_by_name() { - let cases: [(&[(&str, &[&str], &[&str])], _, _); 7] = [ + let cases: [(&[Row<'_>], _, _); 7] = [ (&[("a", &["framebuffer"], &[]), ("b", &["framebuffer"], &[])], ("a", "framebuffer"), ("b", "framebuffer")), (&[("a", &["isa:0060,0064:1,12"], &[]), ("b", &["isa:0060,0064:1,12"], &[])], ("a", "isa:0060,0064:1,12"), ("b", "isa:0060,0064:1,12")), ( From c3f4e070f1bbf515ead7a4666a54615c56bbae1f Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 14 Oct 2026 18:17:04 +0200 Subject: [PATCH 3/6] issues: lifecycle stage 1.2 is done, and the three tracks folded into it say so The track records #926 as stage 1.2 and its present state. The child's-end track's stage 1 (the waiter threads) is done by it; the supervisor track's stage 2 starts from the `toyos-supervisor` crate it made; and the kernel track's step 10 names its usbd as this track's stage 4, a `drives` row. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1 --- ...nt-and-a-parent-takes-its-children-down.md | 7 +-- ...n-is-driven-from-its-arrival-to-its-end.md | 54 ++++++++----------- ...-small-interrupts-post-and-threads-wait.md | 5 +- ...rvisor-is-host-tested-and-owns-the-stop.md | 5 +- 4 files changed, 33 insertions(+), 38 deletions(-) diff --git a/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md b/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md index 4bca0a8ba32..d11a28db35f 100644 --- a/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md +++ b/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md @@ -37,9 +37,10 @@ directory is `issues/every-program-sees-only-the-files-it-was-given.md`'s. ## Stages -1. **init's waiter threads go**. A child's end is readiness on its handle, - and init still parks one thread per service on `SYS_PROCESS_WAIT` - (`close_when_it_ends`, `userland/init/src/main.rs`). *Exit*: init starts no +1. **Done** (#926, stage 1.2 of + `issues/a-device-function-is-driven-from-its-arrival-to-its-end.md`): + **init's waiter threads go**; the supervisor watches each child's end on + its own loop's poller. *Exit*: init starts no thread to wait for a service's end, and `close_when_it_ends` is gone with all it does kept: a `restart` row that ends is started again on the ports it kept; a row without one has its acceptors closed, so a client's next diff --git a/issues/a-device-function-is-driven-from-its-arrival-to-its-end.md b/issues/a-device-function-is-driven-from-its-arrival-to-its-end.md index 50271b0ed4f..945b289d5b9 100644 --- a/issues/a-device-function-is-driven-from-its-arrival-to-its-end.md +++ b/issues/a-device-function-is-driven-from-its-arrival-to-its-end.md @@ -6,13 +6,13 @@ opened: 2026-10-13 # A device function is driven from its arrival to its end -Held by the orchestrator. #910 is stage 1a. +Held by the orchestrator. #910 is stage 1.1 and #926 stage 1.2. -Today the kernel enumerates PCI once (`pcidev::publish`), and the supervisor -mints each claim only when it spawns a `[boot] start` row. A row names a PCI -function by vendor and device, and the kernel refuses the claim `Ambiguous` on -a machine with two. One thread per serving service parks in -`SYS_PROCESS_WAIT`. Nothing reaches userland when a device arrives or leaves. +Today the kernel enumerates PCI once (`pcidev::publish`) and numbers each +function by its arrival. The supervisor mints a `[boot] start` row's claims +when it spawns it, and starts a `drives` row's process on each function it +reads free in the inventory, at boot and after each end of a process it +started. Nothing reaches userland when a device arrives or leaves. **Ruled** (owner, 2026-10-13): @@ -83,30 +83,18 @@ a machine with two. One thread per serving service parks in its handles owed, of every kind, has run on its last thread (`object::Releases`), so `CLAIM_RETURN` and the guest tests' claim retries went. - 2. **Instances per function**, one pull request, which also updates the - three tracks folded here: - - `toyos_abi::inventory::Pci` gains an `arrival` number, never reused, - and a claim names its function by it; `DeviceRequest::Pci`, - `toyos_pci::claimed` and `ClaimError::Ambiguous` go. - - `lib/manifest` owns the matcher grammar (`pci:CC:SS:PP`, and - `pci:CC:SS:PP/VVVV:DDDD`). A `drives` row starts one instance per - matching function the kernel does not drive, is never in - `[boot] start`, may not `serves`, and reaches its consumer through - `receives`. A single-instance `devices` entry matching two functions - refuses that row's start, naming both. - - Two matchers that can match one function are a build refusal naming - both rows, which replaces `one_claimant_per_device` and keeps its - refusal of duplicate class, `isa` and `part:` names. - - The supervisor's lifecycle decisions move into - `system/supervisor/lifecycle`, host-tested. A refused start marks its - function refused until the inventory record changes and counts - against the arrival's restart budget. - - The supervisor watches each child's `Process` handle, and the waiter - threads go. vdagent becomes a `drives` row, and its "no such - function" path goes with it. - - Its guest arm, a q35 profile with two `edu` functions after the - claim-slot pull request, holds two instances up until judged, each - reporting its own function. `test-runner` gains `inventory`. + 2. **Done** (#926): a claim names a PCI function by its inventory + record's `Arrival`, and `toyos_pci::claimed` and `DeviceRequest::Pci` + went; `ClaimError::Ambiguous` stays only for a partition GUID carried + twice. `lib/manifest` owns the matcher grammar, and two entries one + device can answer are a render refusal naming both rows, which replaced + `one_claimant_per_device`. A `drives` row has one process per function + the kernel does not drive and holds no `serves`, `roles` or `devices`. + The decisions are `toyos-supervisor` (`system/supervisor/lifecycle`), + the refusal law among its host tests; the supervisor watches each + child's `Process` handle and its waiter threads went; vdagent is a + `drives` row. `a_drives_row_holds_one_process_per_function` is the + guest arm, on `Profile::HeadlessTwoEdu`. 2. **Functions arrive and leave.** - Readers get a device-set object, minted from `DEVICE`. - The kernel provides native PCIe hot-plug on a port whose host bridge @@ -124,9 +112,9 @@ a machine with two. One thread per serving service parks in ## Folded here - Stage 1 of `issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md` - (the waiter threads) is this track's stage 1.2. + (the waiter threads) was this track's stage 1.2 (#926). - Stage 2 of `issues/the-supervisor-is-host-tested-and-owns-the-stop.md` - extends the crate stage 1.2 makes. Restart policy and the claims a restart - is owed move with stage 1.2. + extends `toyos-supervisor`, the crate stage 1.2 made, which holds restart + policy and the claims a restart is owed. - Step 10 of `issues/the-kernel-is-small-interrupts-post-and-threads-wait.md` is stage 4. diff --git a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md index db16c3fbaed..6585f3bcae1 100644 --- a/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md +++ b/issues/the-kernel-is-small-interrupts-post-and-threads-wait.md @@ -177,7 +177,10 @@ Beside that order, each waiting on something outside it: Each half measures the kernel's lines and its longest interrupts-off and preemption-off windows, by stage 6 step 2's rule. - 10. **usbd**, stage 5's second half: the whole xHCI moves, HID delivered + 10. **usbd**, stage 5's second half, and stage 4 of + `issues/a-device-function-is-driven-from-its-arrival-to-its-end.md`: + usbd is a `drives` row, one process per xHCI controller the kernel + leaves. The whole xHCI moves, HID delivered to the owner of input over the `input` port and mass storage over `toyos-blockring`. A USB keyboard's transitions never reach the kernel, whose keyboard and mouse streams go when their last kernel diff --git a/issues/the-supervisor-is-host-tested-and-owns-the-stop.md b/issues/the-supervisor-is-host-tested-and-owns-the-stop.md index 354a1890436..dcba3a744c8 100644 --- a/issues/the-supervisor-is-host-tested-and-owns-the-stop.md +++ b/issues/the-supervisor-is-host-tested-and-owns-the-stop.md @@ -49,7 +49,10 @@ nothing. `struct Init`, `SpawnInit`, `TimerInit`, `InitPort` and "asks init" all hit). A match goes where it names the program and stays otherwise; third-party text (`tests/corpus/tinycc/`, the C ports) stays. -2. **Decisions in a pure crate.** The supervisor's decisions live in +2. **Decisions in a pure crate.** **Partly done** (#926): the crate exists + (`system/supervisor/lifecycle`) and holds restart policy, the claims a + restart is owed, the function a `devices` matcher names and which + functions a `drives` row has a process on. The supervisor's decisions live in `toyos-supervisor`, with host tests; `system/supervisor` keeps only handles, spawns and the loop. The decisions: namespace and claim selection from a manifest row; the claims a restart is owed; `SysCap` narrowing; From 9029901b5c301c27be5867c17842f2673a0abe4d Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 14 Oct 2026 18:24:24 +0200 Subject: [PATCH 4/6] issues: three records #926 made stale say what is true after it diskserver's NVMe row is a class matcher with QEMU's id, matched by the supervisor, and two matching controllers refuse its start; the class grammar its exit names is `toyos_manifest::device::PciMatch`. `ClaimError::Ambiguous` is a partition GUID carried twice. The 3.8 ms claim keeps its measured figures, with a note that the path no longer exists in that form; it is not re-measured. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1 --- .../a-claims-own-refusals-are-read-by-nothing.md | 3 ++- ...diskserver-claims-qemus-nvme-by-its-pci-id.md | 16 ++++++++-------- ...-t14-lacks-holds-preemption-off-for-3-8-ms.md | 9 +++++++-- 3 files changed, 17 insertions(+), 11 deletions(-) diff --git a/issues/a-claims-own-refusals-are-read-by-nothing.md b/issues/a-claims-own-refusals-are-read-by-nothing.md index 2b5f100c486..ae50f83338f 100644 --- a/issues/a-claims-own-refusals-are-read-by-nothing.md +++ b/issues/a-claims-own-refusals-are-read-by-nothing.md @@ -11,7 +11,8 @@ Refusals of `kernel/src/pcidev/mod.rs` that no test reaches: - `ClaimError::Owned`, `Refusal::NoInterrupt`, `Refusal::CapsTruncated` and the domain's fault, whose guest readers the guest suite's cut moved to stage J of `issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md`; -- `ClaimError::Ambiguous`, a config naming a device this machine has two of; +- `ClaimError::Ambiguous`, a partition GUID two partitions carry (a PCI + claim names one function by its arrival since #926); - `ClaimError::KernelDriven`, a claim on a function one of this kernel's own drivers bound; - `ClaimError::Exhausted`, a claim past the vectors, remapping entries or diff --git a/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md b/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md index 47bfa96fd04..bb64e18908b 100644 --- a/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md +++ b/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md @@ -6,11 +6,11 @@ opened: 2026-10-12 # diskserver claims QEMU's NVMe by its PCI id and no other controller -Both shipped boot configs give diskserver one device, `pci:1b36:0010` -(`images/system.toml`, `images/console.toml`): the vendor and device id of -QEMU's own `nvme` model, under Red Hat's vendor id. A claim matches the id exactly (`pcidev::claim`, -`kernel/src/pcidev/mod.rs`), so on any other NVMe controller the supervisor -endows diskserver nothing, it says `no NVMe controller this row names is on +Both shipped boot configs give diskserver one device, `pci:01:08:02/1b36:0010` +(`images/system.toml`, `images/console.toml`): the NVMe class with the vendor and device id of +QEMU's own `nvme` model, under Red Hat's vendor id. The supervisor matches the id exactly +(`toyos_manifest::device::PciMatch`, `toyos_supervisor::claims`), so on any other NVMe controller it +endows diskserver nothing, diskserver says `no NVMe controller this row names is on this machine; serving no partition`, and DATA, LOG and BOOT are absent unless the image is on a USB stick the kernel drives. The driver is not QEMU's: `Controller::open` (`system/diskserver/src/nvme.rs`) takes any function whose @@ -35,15 +35,15 @@ same for NICs ("init binds drivers to hardware from a declared match table by vendor, device and class"). Two constraints a class row meets that an id row does not: a machine with two -NVMe controllers matches twice, which `pcidev::claim` refuses as `Ambiguous` -today; and a machine whose NVMe holds another operating system, as the T14's +NVMe controllers matches twice, which refuses diskserver's start, naming both +(`toyos_supervisor::claims`); and a machine whose NVMe holds another operating system, as the T14's holds Ubuntu, is handed to diskserver, which then serves only the partitions its table and grants name (`system/diskserver/src/main.rs`). **Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. **Exit**: a `devices` row names a PCI function by class code, which -`toyos_abi::syscall::DeviceRequest::parse` takes and refuses malformed; both +`toyos_manifest::device::PciMatch::parse` takes and refuses malformed (#926); both shipped configs name NVMe that way; and a guest boot whose controller is QEMU's `nvme` with `use-intel-id=on` serves DATA through diskserver, which a row naming `1b36:0010` instead reds. The configs the T14 boots keep their id diff --git a/issues/the-supervisors-claim-of-a-pci-function-the-t14-lacks-holds-preemption-off-for-3-8-ms.md b/issues/the-supervisors-claim-of-a-pci-function-the-t14-lacks-holds-preemption-off-for-3-8-ms.md index f1fc100411d..38f6284e366 100644 --- a/issues/the-supervisors-claim-of-a-pci-function-the-t14-lacks-holds-preemption-off-for-3-8-ms.md +++ b/issues/the-supervisors-claim-of-a-pci-function-the-t14-lacks-holds-preemption-off-for-3-8-ms.md @@ -12,9 +12,14 @@ every 2 ms (#681, comment 5968784527) read init's `SYS_DEVICE_CLAIM` for blockd's `pci:1b36:0010`, which this machine does not have, holding cpu0's interrupts off for 3,817,265 ns at 1.162 s, from the syscall's entry to its return, with `MSR_SMI_COUNT` unmoved. Its one sample, 1.7 µs before it closed, -is in `PciDevice::is_id` under `pcidev::claim`. By reading, the claim reads the +is in `PciDevice::is_id` under `pcidev::claim`. By reading, the claim then read the vendor ID of each of the 24 functions the kernel enumerated -(`kernel/src/pcidev/mod.rs`); nothing has said what it spends 3.8 ms on. +(`kernel/src/pcidev/mod.rs`); nothing has said what it spent 3.8 ms on. + +**That path no longer exists in that form** (#926, not re-measured): the +supervisor matches a `devices` entry against the inventory and asks the kernel +for no function the machine lacks, and `pcidev::claim` takes an arrival and +reads no function's vendor ID. The figures here stay as measured. A syscall's body now runs with interrupts open and preemption off (`issues/syscall-preemption-is-incidental.md`). #716's interleaved From 28360a6e53fcdbaccaf07df1d1af745b383cb370 Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 14 Oct 2026 18:40:57 +0200 Subject: [PATCH 5/6] Review: the manifest spells its own card id, and only WouldBlock means a child runs B1: `PciMatch` parses its `VVVV:DDDD` card with `lib/manifest`'s own four-lowercase-digit rule, and the spelling test moved there. `PciId::parse` stays in the ABI as the `xhci-leave=` parameter's spelling alone, which its doc now says. B2: `exit_of` reads a wait without parking through `toyos_supervisor::restart::exited`: `WouldBlock` is a process still running, an exit code its end, and any other answer on the supervisor's own handle panics, as the deleted waiter did. Host-tested, both arms. B3: `DeviceRequest`'s doc says only what reads its spelling. S1: the drives arm counts the kernel's hand-overs and the holders' lines as lists, so a function handed over twice is red. S3: a `drives` row named in another row's `starts` is a render refusal, as one in `[boot] start` is: a launch would start it holding no function. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1 --- lib/manifest/src/device.rs | 22 +++++++++++++++--- lib/manifest/src/lib.rs | 6 ++++- sdk/abi/src/syscall.rs | 25 ++++++++------------- system/supervisor/lifecycle/src/restart.rs | 26 ++++++++++++++++++++++ system/supervisor/src/main.rs | 4 ++-- tests/toyos.rs | 10 ++++++--- 6 files changed, 68 insertions(+), 25 deletions(-) diff --git a/lib/manifest/src/device.rs b/lib/manifest/src/device.rs index 17ee978c805..6fecd26fcea 100644 --- a/lib/manifest/src/device.rs +++ b/lib/manifest/src/device.rs @@ -59,7 +59,10 @@ impl PciMatch { pub fn parse(text: &str) -> Option { let rest = text.strip_prefix(Self::PREFIX)?; let (code, card) = match rest.split_once('/') { - Some((code, card)) => (code, Some(PciId::parse(card)?)), + Some((code, card)) => { + let (vendor, device) = card.split_once(':')?; + (code, Some(PciId { vendor: hex16(vendor)?, device: hex16(device)? })) + } None => (rest, None), }; let mut class = [0u8; 3]; @@ -93,11 +96,20 @@ impl fmt::Display for PciMatch { /// Exactly two lowercase hex digits: one spelling per value, because the /// label a claim is endowed under is the entry's text. fn hex8(text: &str) -> Option { + lower_hex(text, 2).and_then(|v| u8::try_from(v).ok()) +} + +/// Exactly four lowercase hex digits, for [`hex8`]'s reason. +fn hex16(text: &str) -> Option { + lower_hex(text, 4).and_then(|v| u16::try_from(v).ok()) +} + +fn lower_hex(text: &str, digits: usize) -> Option { let lowercase = text.bytes().all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)); - if text.len() != 2 || !lowercase { + if text.len() != digits || !lowercase { return None; } - u8::from_str_radix(text, 16).ok() + u32::from_str_radix(text, 16).ok() } #[cfg(test)] @@ -140,6 +152,10 @@ mod tests { "pci:01:08", "pci:01:08:02:00", "pci:1:08:02", "pci:01:08:0A", "pci:01:08:02/", "pci:01:08:02/1B36:0010", "pci:01:08:02/1b36", "pci:01:08:02/1b36:0010/x", "pci:+1:08:02", "pci:", "pci", + // The card is four lowercase hex digits each and nothing else: two + // spellings would be two labels for one claim. + "pci:02:00:00/1Af4:1041", "pci:02:00:00/1af4:41", "pci:02:00:00/1af4", "pci:02:00:00/1af4:1041:", + "pci:02:00:00/:", "pci:02:00:00/zzzz:1041", "pci:02:00:00/1af4:104g", "pci:02:00:00/+af4:1041", ] { assert_eq!(PciMatch::parse(bad), None, "{bad} parsed"); assert_eq!(Entry::parse(bad), None, "{bad} parsed as an entry"); diff --git a/lib/manifest/src/lib.rs b/lib/manifest/src/lib.rs index ebf1978a28f..b4a87587e2e 100644 --- a/lib/manifest/src/lib.rs +++ b/lib/manifest/src/lib.rs @@ -404,7 +404,9 @@ pub enum RenderError { /// would be refused at boot, or two rows would race for one function. Overlap { first: (String, String), second: (String, String) }, /// A `drives` row that also has what one process per function cannot - /// carry: `serves`, `roles` or `devices`, or a place in `[boot] start`. + /// carry: `serves`, `roles` or `devices`, or a place in `[boot] start` or + /// another row's `starts`, either of which would start it holding no + /// function. Drives { program: String, also: &'static str }, } @@ -499,6 +501,7 @@ fn claims_apart(manifest: &Manifest) -> Result<(), RenderError> { (!program.roles.is_empty(), "roles"), (!program.devices.is_empty(), "devices"), (manifest.start.contains(&program.name), "[boot] start"), + (manifest.programs.iter().any(|row| row.starts.contains(&program.name)), "starts"), ]; if let Some((_, also)) = also.into_iter().find(|(has, _)| *has) { return Err(RenderError::Drives { program: program.name.clone(), also }); @@ -999,6 +1002,7 @@ mod tests { assert_eq!(drives(|m| m.programs[0].roles = vec!["data".into()]), refused("roles")); assert_eq!(drives(|m| m.programs[0].devices = vec!["keyboard".into()]), refused("devices")); assert_eq!(drives(|m| m.start = vec!["a".into()]), refused("[boot] start")); + assert_eq!(drives(|m| m.programs[0].starts = vec!["a".into()]), refused("starts")); } #[test] diff --git a/sdk/abi/src/syscall.rs b/sdk/abi/src/syscall.rs index 797dc7408eb..e590afc7933 100644 --- a/sdk/abi/src/syscall.rs +++ b/sdk/abi/src/syscall.rs @@ -1363,8 +1363,9 @@ pub struct PciId { } impl PciId { - /// `":"`, four lowercase hex digits each; one spelling per - /// function, so that comparing the strings compares the functions. + /// `":"`, four lowercase hex digits each: the kernel's + /// `xhci-leave=` boot parameter. A manifest matcher's card is spelled by + /// `toyos-manifest`'s own rule. pub fn parse(text: &str) -> Option { let (vendor, device) = text.split_once(':')?; Some(Self { vendor: hex16(vendor)?, device: hex16(device)? }) @@ -1535,10 +1536,8 @@ fn decimal_line(text: &str) -> Option { /// ports and lines. /// /// Every request but a PCI function's is also a `devices` entry, spelled once -/// here, because four places read that spelling — the build system's gate, -/// `/system/bin/supervisor`'s mint, the kernel's claim and the claimant's own -/// lookup — and a second would be a name a config can write and a program -/// cannot find. +/// here ([`Self::parse`], [`Self::write_name`]): the manifest that names it and +/// the claimant that looks its claim up by that name read one spelling. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum DeviceRequest { Class(DeviceType), @@ -2744,17 +2743,11 @@ mod tests { modules(&answer).for_each(drop); } - /// Four lowercase hex digits each and nothing else, because two spellings - /// would be two names for one card: `1af4:41` beside `1af4:0041`, and - /// `1AF4:1041` beside `1af4:1041`. + /// `xhci-leave=`'s id, four lowercase hex digits each. #[test] - fn a_pci_id_is_four_lowercase_hex_digits_each_and_nothing_else() { - let want = PciId { vendor: 0x1af4, device: 0x1041 }; - assert_eq!(PciId::parse("1af4:1041"), Some(want)); - for bad in [ - "1AF4:1041", "1Af4:1041", "1af4:41", "1af4", "1af4:1041:", "", ":", "zzzz:1041", - "1af4:104g", - ] { + fn the_xhci_leave_parameter_is_an_id_in_four_lowercase_hex_digits_each() { + assert_eq!(PciId::parse("8086:9a13"), Some(PciId { vendor: 0x8086, device: 0x9a13 })); + for bad in ["8086:9A13", "8086:9a1", "8086", "8086:9a13:"] { assert_eq!(PciId::parse(bad), None, "{bad:?} parsed"); } } diff --git a/system/supervisor/lifecycle/src/restart.rs b/system/supervisor/lifecycle/src/restart.rs index 032a31a9e44..fc49f652575 100644 --- a/system/supervisor/lifecycle/src/restart.rs +++ b/system/supervisor/lifecycle/src/restart.rs @@ -3,6 +3,7 @@ use std::collections::VecDeque; use std::time::{Duration, Instant}; +use toyos_abi::syscall::SyscallError; use toyos_manifest::Program; /// The most starts a service, or one function's processes, spends inside @@ -31,6 +32,17 @@ impl Budget { } } +/// A wait without parking on a process of the supervisor's own, as an exit +/// code once it has ended. `WouldBlock` is the one answer that means it runs: +/// any other refusal of a handle the supervisor holds is its own bug. +pub fn exited(answer: Result) -> Option { + match answer { + Ok(exit) => Some(exit), + Err(SyscallError::WouldBlock) => None, + Err(e) => panic!("supervisor: its own process handle refused a wait: {e:?}"), + } +} + /// Whether `row`'s process that ended with `exit` is owed another: the row is /// `restart`, and it ended other than by exit 0, which a program says only /// when it has nothing to serve. @@ -92,6 +104,20 @@ mod tests { assert_eq!(after_end(&row(true), 1, &mut budget, later), AfterEnd::Restart); } + #[test] + fn a_process_runs_only_while_its_wait_would_block() { + assert_eq!(exited(Ok(3)), Some(3)); + assert_eq!(exited(Err(SyscallError::WouldBlock)), None); + } + + /// Read as "still running", a refused wait would wake the loop on every + /// pass and act on nothing. + #[test] + #[should_panic(expected = "refused a wait")] + fn any_other_answer_to_a_wait_is_the_supervisors_own_bug() { + exited(Err(SyscallError::PermissionDenied)); + } + /// An end owed nothing spends nothing. #[test] fn an_end_owed_nothing_spends_no_start() { diff --git a/system/supervisor/src/main.rs b/system/supervisor/src/main.rs index 3b2c5079dce..309f2ae2530 100644 --- a/system/supervisor/src/main.rs +++ b/system/supervisor/src/main.rs @@ -2450,9 +2450,9 @@ fn pci_at(at: PciAddr) -> String { format!("{:02x}:{:02x}.{}", at.bus, at.dev, at.func) } -/// The exit code of `child` once it has ended. +/// The exit code of `child` once it has ended ([`restart::exited`]). fn exit_of(child: &Child) -> Option { - toyos_abi::syscall::process_wait_nonblock(toyos::RawHandle(child.as_raw_handle())).ok() + restart::exited(toyos_abi::syscall::process_wait_nonblock(toyos::RawHandle(child.as_raw_handle()))) } /// Where the acceptors of the names a started program serves come from. diff --git a/tests/toyos.rs b/tests/toyos.rs index 5745cc705c9..748bf0beeda 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -3910,11 +3910,15 @@ fn a_drives_row_holds_one_process_per_function() -> Result<(), String> { .map_err(|e| format!("{e}\n{console}"))?; let said = serial::Serial::named("the drives boot", console.clone()); said.must_be_clean()?; - let handed: BTreeSet<&str> = + let mut handed: Vec<&str> = console.lines().filter_map(|l| l.split_once("pcidev: PCI ")?.1.split_once(HANDED).map(|(at, _)| at)).collect(); - let holding: BTreeSet<&str> = + let mut holding: Vec<&str> = console.lines().filter_map(|l| l.split_once(HOLDING)?.1.split_whitespace().next()).collect(); - if handed.len() != 2 || holding != handed { + handed.sort_unstable(); + holding.sort_unstable(); + // Each function handed over once, to one holder: a list, so a second + // hand-over or a second holder of one function is red. + if handed.len() != 2 || handed[0] == handed[1] || holding != handed { return Err(format!("the kernel handed over {handed:?} and the holders say they hold {holding:?}:\n{console}")); } let judged = job_window(&mut qemu, "test_rs_drives_judged")?; From 8d9503f47bff37f0f6d9cf75434471c1e331928d Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 14 Oct 2026 18:54:23 +0200 Subject: [PATCH 6/6] issues: the 3.8 ms claim says what replaced it, unmeasured, and that its exit reads a claim no boot makes Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1 --- ...unction-the-t14-lacks-holds-preemption-off-for-3-8-ms.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/issues/the-supervisors-claim-of-a-pci-function-the-t14-lacks-holds-preemption-off-for-3-8-ms.md b/issues/the-supervisors-claim-of-a-pci-function-the-t14-lacks-holds-preemption-off-for-3-8-ms.md index 38f6284e366..aaaf13ef647 100644 --- a/issues/the-supervisors-claim-of-a-pci-function-the-t14-lacks-holds-preemption-off-for-3-8-ms.md +++ b/issues/the-supervisors-claim-of-a-pci-function-the-t14-lacks-holds-preemption-off-for-3-8-ms.md @@ -19,7 +19,11 @@ vendor ID of each of the 24 functions the kernel enumerated **That path no longer exists in that form** (#926, not re-measured): the supervisor matches a `devices` entry against the inventory and asks the kernel for no function the machine lacks, and `pcidev::claim` takes an arrival and -reads no function's vendor ID. The figures here stay as measured. +reads no function's vendor ID. The figures here stay as measured. What +replaced the claim at boot is one `SYS_DEVICE_INVENTORY` read per start of a +row with a PCI matcher, and nobody has measured it; the exit below reads a +claim no boot now makes, so a T14 reading of that read is what decides +whether this closes. A syscall's body now runs with interrupts open and preemption off (`issues/syscall-preemption-is-incidental.md`). #716's interleaved