issues: the speed track's stages stand on the vendors' documents and the literature, and report their figures

issues/a-wake-to-a-busy-cpu-waits-for-its-next-pass-and-no-idle-cpu-can-take-it.md

@@ -38,10 +38,36 @@ dispatch on the kernel clock; quantiles are bucket upper bounds, accurate to
 With 7 spinners every wake to a busy target waited out a whole quantum on
 both machines, while the slowest wake to a sleeping one took 2.16 µs (T14)
 and 3.17 µs (AMD bench), and the pair fell from about 181,600 (T14) and
-205,200 (AMD bench) rounds a second with no spinners to 100. The kill condition fixed before the run, the change
-dropped if the busy-target p99 is under about 100 µs with 7 and with 8
-spinners on both benches, did not fire: it reads 68 to 105 times that.
+205,200 (AMD bench) rounds a second with no spinners to 100.
+
+With 7 spinners this is the Overload-on-Wakeup bug of Lozi et al., "The Linux
+Scheduler: a Decade of Wasted Cores" (EuroSys 2016, §3.3): a thread that was
+asleep wakes on an overloaded core while other cores are idle. Its fix wakes
+the thread on the core it last ran on if that core is idle, and otherwise on
+the core idle longest, where one is; the paper applies it only where the power
+policy keeps cores out of low-power states, since waking a core from one costs
+power. The 7-spinner arm does not take that condition: a wake goes to an idle
+core in whatever state it waits, so once
+`issues/idle-cpus-enter-the-c-states-cst-names.md` lets cores sleep deeper,
+the 100 µs row's p99 also holds the woken core's exit from its state, and the
+row names the states its idle CPUs entered. With 8 spinners no core idles, and
+a wake onto a CPU running a task is the fair band's pick to decide, which is
+the vruntime/lag family (`kernel/pure/sched/fair.rs:1-3`) of EEVDF. There the
+resource is "allocated in time quanta of size at most q. At the beginning of
+each time quantum a client is selected", and "A new quantum is allocated to
+the client which has the eligible request with the earliest virtual deadline"
+(Stoica et al., "A Proportional Share Resource Allocation Algorithm for
+Real-Time, Time-Shared Systems", RTSS 1996, the algorithm of Stoica and
+Abdel-Wahab's TR-95-22), so a task woken inside a quantum is picked no sooner
+than the next one's start. That arm follows it with a wake's q the protected
+slice, 1 ms (`PROTECTED_RUN_NS`, https://github.com/ToyOSOrg/ToyOS/pull/874,
+not on `main`, so that arm is blocked on #874), so its test is that a woken
+task runs within one protected slice, and its p99 against 100 µs is reported.
+With 7 spinners the 100 µs bound is the exit's test that the stage works, not
+the judge of its design.
 
 **Exit**: a metal row on the T14 and the AMD bench that wakes a task onto a
-CPU running a CPU-bound thread, with 7 and with 8 such threads on 8 CPUs, and
-holds its post-to-dispatch p99 under 100 µs.
+CPU running a CPU-bound thread, with 7 and with 8 such threads on 8 CPUs:
+with 7 it holds the post-to-dispatch p99 under 100 µs, and with 8 it holds
+that a woken task runs within one protected slice and reports its p99, once
+#874 has landed `PROTECTED_RUN_NS`.

issues/diskserver-drives-nvme-apst.md

@@ -7,12 +7,21 @@ opened: 2026-09-29
 # blockd drives NVMe APST
 
 After blockd drives the T14's drive, the blockd step of
-`issues/the-kernel-is-small-interrupts-post-and-threads-wait.md`. As
-Linux at `Ubuntu-6.8.0-142.142`, APST stays within
-`default_ps_max_latency_us` and honours `NVME_QUIRK_NO_APST` and
-`NVME_QUIRK_NO_DEEPEST_PS` (`drivers/nvme/host/core.c:63,2495-2598`).
+`issues/the-kernel-is-small-interrupts-post-and-threads-wait.md`. The NVMe
+base specification defines APST: enabled through Feature 0Ch, the controller
+moves itself, once idle for the time a power state's table entry names, to
+the non-operational state that entry names, one with no higher idle power;
+and the drive reports each state's worst entry and exit latency, a
+transition costing the old state's exit latency plus the new one's entry
+latency (NVM Express Base Specification 2.1, §5.1.25.1.6, §8.1.17 and
+§8.1.17.2). blockd builds the table from the power state descriptors of the
+drive's own `id-ctrl`. The specification bounds no latency: blockd's bound is
+its design round's, chosen on its merits. It follows the specification
+with no table of drives: a drive that misbehaves under it is a defect,
+measured on the bench and filed.
 
-**Exit**: host tests of the APST table; on the T14, the one proving machine
-with a physical NVMe drive, the block figures and the idle minute with the
-drive idle. **Mutation**: a table that admits a state past the bound.
-**Oracle**: Linux's `nvme_configure_apst` over the T14's `id-ctrl`.
+**Exit**: host tests of the APST table; on the
+T14, the one proving machine with a physical NVMe drive, the block figures and
+the idle minute with the drive idle. **Mutation**: a table that admits a state
+past the bound. **Oracle**: the specification's rules for the table over the
+T14's `id-ctrl`.

issues/fsgsbase-is-taken.md

@@ -0,0 +1,36 @@
+---
+status: open
+kind: track
+opened: 2026-09-29
+---
+
+# FSGSBASE is taken
+
+`CR4.FSGSBASE` is forbidden (`kernel/src/arch/x86_64/control_regs.rs:149-153`):
+GS always holds the kernel's per-CPU base and the kernel runs no `swapgs`.
+Taking it costs a `swapgs` entry and the swapgs barriers of
+`issues/user-pointer-checks-have-no-spectre-v1-fence-and-smap-is-optional.md`.
+Intel names what it gives, user programs' FS and GS bases without a system
+call and a more efficient kernel context switch, and has it used "never in
+production environments unless accompanied by a complete OS implementation":
+one whose context switch and process-trace system calls manage the FS and GS
+bases apart from their selectors, and whose ring transitions are insensitive
+to the GS base (Guidance for Enabling FSGSBASE, ID 767615). AMD documents the
+four instructions, enabled by CR4.FSGSBASE at every privilege level, and
+recommends nothing on their use (APM 24594 rev. 3.36, Vol. 3, RDFSBASE and
+WRFSBASE). ToyOS takes it under Intel's condition: its switch saves and
+restores each thread's FS and GS bases as the thread left them, its ring
+transitions are insensitive to the GS base they find, and it has no system
+call that reads or writes another thread's registers
+(`sdk/abi/src/syscall.rs`). Its figure is reported and decides nothing.
+
+**Exit**: on every proving machine and under TCG, a `boot-actuators` arm
+enters vector 2 at CPL 0 while the user GS base is live, between entry and its
+`swapgs` or after the exit path's, and the handler finds the kernel's per-CPU
+data; a thread's FS and GS bases, written with WRFSBASE and WRGSBASE, read
+back unchanged after its CPU has run another thread; `CR4_FORBIDDEN` is
+deleted, and so is `gsbase_locked` with its `user-writable-gsbase` kernel from
+stage C of `issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md`;
+the T14's pipe round trips read with and without FSGSBASE. **Mutation**: an
+IST entry that decides `swapgs` from the saved CS alone. **Oracle**: the
+vector-2 arm on the T14.

issues/idle-is-hlt-so-every-wake-of-an-idle-cpu-is-an-ipi.md

@@ -12,13 +12,33 @@ doorbell (`kernel/pure/sched/mailbox.rs:474-490`) and then sends a kick IPI
 (`kernel/src/arch/x86_64/apic.rs:256-260`), whose entry and handler run on the
 woken CPU before its `hlt` returns (`kernel/src/arch/x86_64/idt/kick.rs:3-9`).
 A CPU that waits in MWAIT on its doorbell's line (CPUID.01H:ECX[3]) is woken
-by the store alone and takes no interrupt. The T14 enumerates it (`monitor`,
-`kernel/cpuvuln/fixtures/t14/cpuinfo.txt:20`); whether the AMD bench does is
-unread. `issues/idle-cpus-enter-the-c-states-cst-names.md` is which state the
+by the store alone and takes no interrupt. The SDM names the IPI a halted CPU
+needs as a delay in serving new work and has an OS consider MONITOR/MWAIT in
+place of `hlt` in its C1 idle loop (325462-093, Vol. 3A §11.10.4,
+§11.10.6.4), and Intel's optimization manual names the C1 loop's
+responsiveness as what that buys (248966-051, Vol. 1 §11.4). AMD's APM
+documents MWAIT as a wait for another processor's store to the range MONITOR
+set up, sized by CPUID 0000_0005h (24594 rev. 3.36, Vol. 3, MONITOR and
+MWAIT), and neither it, its Vol. 2 (24593 rev. 3.42) nor its optimization
+guide for family 17h models 30h and greater (56305 rev. 3.02) names an idle
+loop. Nor does the AMD bench's own reference: it enumerates MONITOR/MWAIT
+unless firmware disables them, gives the monitor line as 64 bytes, and has
+MWAIT support one C1 sub-state (PPR 55922 rev. 3.06, `CPUID_Fn00000001_ECX`
+bit 3, `CPUID_Fn00000005`). So the stage is backed on Intel by §11.10.4 and
+§11.10.6.4, and is Intel's alone: the AMD bench keeps `hlt` and the kick, and
+whether it takes the doorbell is the orchestrator's to decide once that bench's
+`_CST` is read. A 2020+ Intel CPU whose specification update names an erratum
+by which a store may not wake MWAIT keeps the kick. The T14's lists no MONITOR
+or MWAIT erratum (631123-034); Intel's Core Ultra 200V specification update is
+unread. Whether the T14 and the AMD bench enumerate MONITOR/MWAIT is ToyOS's
+own read to make, and no boot prints CPUID.01H:ECX yet.
+`issues/idle-cpus-enter-the-c-states-cst-names.md` is which state the
 wait enters; this file is what ends it. The doorbell shares a line with its
 neighbours' published counters
 (`issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md`),
-so a monitor on it today would also wake on their writes.
+so a monitor on it today would also wake on their writes; the SDM has
+monitored data padded to the largest monitor line size CPUID.05H reports
+(§11.10.5).
 
 **Measured** (`wt/toyos-scout-tlb` at `d4c2f8ebe`, `main`'s `177e72a9e` plus
 the scout; https://github.com/ToyOSOrg/ToyOS/pull/866#issuecomment-6119443198).
@@ -36,32 +56,13 @@ send to the woken task running, p50 in TSC ticks per direction:
 A CPU woken from MWAIT by the store skips all three segments but pays its own
 wake from the store, which no run has timed and which delivery's C1 exit
 stands in for here, so the share is an upper bound on what the doorbell
-saves. Applied to that bound, the line fixed before the scout, the doorbell
-dropped under 20 % of the wake, fires on the T14 and not on the AMD bench.
+saves.
 
-**Next: an MWAIT twin** beside a control arm, both timed by the same split in
-the same session. The control arm is `main` plus the scout with the doorbell
-on a line of its own, which is
-`issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md`'s
-change. The twin is the control arm with each idle CPU waiting in MWAIT on its
-doorbell's line, hint C1 (EAX = 0), the state `hlt` enters, and the wake's
-kick not sent. The two arms differ only in what ends the wait, which is what
-the bound above covers. Its kill line, fixed here before it runs: the
-doorbell is built ahead of `issues/idle-cpus-enter-the-c-states-cst-names.md`
-only if, on a proving machine that enumerates MONITOR/MWAIT, the twin's whole
-idle wake p50 is at least 20 % under the control arm's in both directions.
-The T14's bound is under that line, so the twin runs on the AMD bench, and
-reads CPUID.01H:ECX[3] there first; if the AMD bench does not enumerate it, no
-proving machine can clear the line. Its clock control, fixed with the line:
-each arm reads APERF and MPERF on CPUs 4 and 5 before and after the split,
-and where the two arms' ratios of APERF's advance to MPERF's differ by more
-than 1 % on either CPU, both arms' readings are void and decide nothing.
-
-**Exit**: the twin's reading, posted, and an idle CPU on every proving machine
-that enumerates MONITOR/MWAIT woken by its doorbell's store with no IPI. Where
-the twin clears the line, that is built ahead of the C-states. Where it does
-not, this file stays open behind
-`issues/idle-cpus-enter-the-c-states-cst-names.md`: the 20 % prices an idle
-path beside `hlt`, which that track builds anyway, and once an idle CPU waits
-in MWAIT there, putting its monitor on the doorbell's line and sending no
-kick adds no second path.
+**Exit**: a boot line printing CPUID.01H:ECX, and a T14 boot carrying it
+cited for bit 3; an idle CPU on every Intel proving machine that enumerates
+MONITOR/MWAIT woken by its doorbell's store with no IPI, and the split above
+read again on the T14 after it; the Core Ultra 200V specification update read,
+and where it or any other 2020+ Intel specification update names a MONITOR or
+MWAIT erratum, the kick kept on that CPU by a verdict that is a pure function
+of CPUID's vendor, family and model, host-tested with that row; and the AMD
+bench on `hlt` and the kick.

issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md

@@ -40,13 +40,20 @@ the parts it links):
   only these four; and the load event (D2H/04H) on this CPU also counts clean
   forwards.
 
-The kill condition fixed before the run, these recorded as structure only if
-both figures are under 1 %, did not fire on its HITM half, so the contention
-is not dismissed. Nothing in the run attributes a miss to the `BITMAP` lock,
-`IN_PASS`, `CpuHandle` or `Frontier`, and the AMD bench was not read. The
-next measurement attributes the HITM count to the lines it lands on, on both
+Intel's optimization manual has critical data and locks allocated apart,
+aligned to no less than the false-sharing threshold, the CLFLUSH line size of
+CPUID.01H:EBX[15:8] (248966-051, Vol. 1 §11.4.5, User/Source Coding Rule
+17), which backs the stage on the T14. On the AMD bench the stage stands on
+ToyOS's rule that per-CPU data goes on lines of its own, resting on the line
+AMD documents: 8 quadwords by CPUID.01H:EBX[15:8] (PPR 55922 rev. 3.06,
+`CPUID_Fn00000001_EBX[15:8]`), and 64 bytes (Software Optimization Guide for
+AMD Family 17h Processors, 55723 rev. 3.00, §2.6.1).
+
+Nothing in the run attributes a miss to the `BITMAP` lock, `IN_PASS`,
+`CpuHandle` or `Frontier`, and the AMD bench was not read. The next
+measurement attributes the HITM count to the lines it lands on, on both
 benches.
 
-**Exit**: that attribution, posted; and each of the four either written per
-CPU on a line no other CPU writes, or kept with its refusal reason at its
-site and its measured cost on both benches in the closing commit.
+**Exit**: that attribution, posted, and read again after the change; and
+each of the four either written per CPU on a line no other CPU writes, or
+kept with its refusal reason at its site.

issues/the-timer-is-a-lapic-count-re-armed-every-pass-never-a-tsc-deadline.md

@@ -20,9 +20,20 @@ https://github.com/ToyOSOrg/ToyOS/pull/861#issuecomment-6113257693). A timer fir
 calibration spins the boot for 10 ms (`apic.rs:315-320`, from
 `kernel/src/arch/x86_64/boot.rs:135`). TSC-deadline mode (CPUID.01H:ECX[24])
 takes the absolute deadline in TSC units as one `IA32_TSC_DEADLINE` write,
-with no calibration of a second timebase. The T14 enumerates it
-(`tsc_deadline_timer`, `kernel/cpuvuln/fixtures/t14/cpuinfo.txt:20`); whether
-the AMD bench does is unread.
+with no calibration of a second timebase (SDM 325462-093, Vol. 3A §13.5.4.1).
+Whether the T14 enumerates it is ToyOS's own read to make, and no boot prints
+CPUID.01H:ECX yet. AMD documents no such mode: its
+CPUID reference has CPUID.01H:ECX[24] reserved (APM 24594 rev. 3.36, Vol. 3,
+§E.3.2), and so does the AMD bench's own (PPR 55922 rev. 3.06,
+`CPUID_Fn00000001_ECX`), so the AMD bench keeps the count, its calibration
+and its spin.
+
+**Its speed reading decides it.** The SDM defines the mode and recommends it
+over the count nowhere, and AMD documents none, so nothing published covers
+the choice. Intel's specification update for the T14's processor lists no
+TSC, TSC-deadline or APIC-timer erratum (631123-034, the UP3 line, CPUID
+000806C1h). Nothing but speed is its purpose: a deadline removes no path,
+since a CPU without one keeps the count.
 
 **Measured** (`wt/toyos-scout-tlb` at `d4c2f8ebe`, `main`'s `177e72a9e` plus
 the scout; https://github.com/ToyOSOrg/ToyOS/pull/866#issuecomment-6119443198),
@@ -48,22 +59,21 @@ is an upper bound on what deadline mode saves, which still writes once to arm
 and once to stop; on the AMD bench both sit at its TSC's floor.
 
 **Next: a TSC-deadline twin on the T14**, `main` plus the scout with the
-one-shot armed and stopped through `IA32_TSC_DEADLINE`, timed by the same
-split beside `main` in the same session, with the re-arm segment opening at
-`set_timer` (`kernel/src/hw.rs:49`) in both arms. Its kill line, fixed here
-before it runs: deadline mode is not worth building for speed if what it
-takes off a pass, `main`'s re-arm p50 less the twin's, is under 5 % of
-`main`'s pick plus switch in either direction (221 and 216 ticks on the
-scout's figures). Its clock control, fixed with the line: each arm reads
-APERF and MPERF on CPUs 4 and 5 before and after the split, and where the two
-arms' ratios of APERF's advance to MPERF's differ by more than 1 % on either
-CPU, both arms' readings are void and decide nothing. Where the line fires,
-this file becomes `rejected` with both readings, and the boot's 10 ms
-calibration spin stays the count's price.
+one-shot armed and stopped through `IA32_TSC_DEADLINE`, beside `main` plus
+the scout in the same session. Each arm reads the 1 ms timer lateness of
+`issues/no-program-measures-toyos-against-linux-on-one-machine.md` and the
+split above, its re-arm segment opening at `set_timer` (`kernel/src/hw.rs:49`)
+in both. Each arm runs 11 times; a run's figure is the median lateness of
+its 1 ms timers, and an arm's is the median of its 11 with their spread,
+lowest to highest. The deadline measures better only where its median is the
+lower and the two spreads do not overlap; where they overlap, the count, the
+path every CPU has, is kept. The split is reported. The twin is a measurement
+arm, deleted once it has answered.
 
-**Exit**: the twin's reading, posted, and the AMD bench's CPUID.01H:ECX[24]
-read. Where the twin clears the line, a deadline is armed on every CPU that
+**Exit**: a boot line printing CPUID.01H:ECX, and a T14 boot carrying it
+cited for bit 24; the twin's reading, posted, and the twin deleted. Where the
+deadline's lateness measures better, a deadline is armed on every CPU that
 enumerates TSC-deadline as one `IA32_TSC_DEADLINE` write of the absolute
-value, and the T14's 1 ms timer lateness of
-`issues/no-program-measures-toyos-against-linux-on-one-machine.md` is no worse
-than the count's.
+value, and the split above is read again after it; where it does not, this
+file becomes `rejected` with both readings, and the boot's 10 ms
+calibration spin stays the count's price.