The scheduler simulator runs at real machine shapes and holds work conservation as an invariant

issues/placement-is-blind-to-caches-and-topology.md

@@ -27,3 +27,10 @@ placement story has three parts none of which we have:
 Composes with the reservation model (placement decides *where*, reservations
 decide *how much*) and with the scheduler simulator (`kernel/sim`), which would need a topology
 model to gate any of this.
+
+The simulator models the tree before the kernel has one, so it compares the
+keys itself: `Shape::level` (`kernel/sim/src/topology.rs`), and the CPUID
+decoding of `scouted` (`kernel/sim/tests/topology.rs`), which reads the benches'
+recorded words. The stage that lands the kernel's `Topology::from_keys` and its
+CPUID decoder moves both onto them in the same change, or the simulator keeps a
+second tree and a second decoder beside the kernel's.

issues/the-push-bounds-an-idle-cpus-wait-only-for-one-victim.md

@@ -0,0 +1,26 @@
+---
+status: open
+kind: finding
+opened: 2026-10-12
+---
+
+# The push bounds an idle CPU's wait only for one victim
+
+The simulator's invariant I15 (`kernel/sim/src/invariants.rs`,
+`work_conservation_bound`) holds the shipped balance path to `cpus + 2`
+protocol rounds of an idle CPU beside stealable work. The bound is derived for
+one victim: that victim passes once a round and its push walks a cursor over
+every sleeper, so the idle CPU is rung within `cpus − 1` of its passes.
+
+With several victims nothing derives a bound. Each victim walks its own cursor,
+so a sleeper can be passed over by each in turn, and a thief probes the CPU
+publishing the most surplus, so a probe sent to a victim that runs dry before it
+answers leaves the thief waiting for a ring again. Neither is shown to happen:
+over the suite at every simulator shape, and over storm and interactive
+workloads at 8, 16 and 64 CPUs, the worst stretch measured is 17 rounds against
+a bound of 66. It is the derivation that is missing, not a schedule that
+crosses it.
+
+Exit: stage S2 of the scheduler design (#872) replaces the most-surplus victim
+with two random choices and the cursor push with a nearest-first one, and
+derives I15's bound for that protocol over any number of victims.

kernel/sim/corpus/old_migrate_kept_the_corpse_i14.trace

@@ -4,5 +4,7 @@
 # note: InTransit(CpuId(1)). soundserver boost-wakes a client thread whose process
 # note: has just been torn down, on the CPU soundserver itself is running, with a
 # note: sibling asleep: place() forwards the corpse instead of reaping it.
-1 2 2 0 1 1 1 1 2 0 0 1 1 1 1 1 0 0 2 1 2 1 2 1 0 1 0 0 2 2 1 2
-0 1 3 1 1 1 0 3 
+# note: re-shrunk after an awake CPU with nothing loaded came to owe its pass at
+# note: once, which renumbers every decision.
+1 1 1 2 0 0 0 0 1 2 2 2 0 1 1 2 0 1 2 0 1 0 1 0 0 3 1 2 0 0 2 0
+2 0 0 0 2 0 2 2 1 1 0 3 

kernel/sim/src/explore.rs

@@ -11,6 +11,7 @@ use kernel::sched::hw::CpuId;
 use crate::choice::ChoiceStream;
 use crate::invariants;
 use crate::latency::{ReadyCause, RunWait};
+use crate::topology::LevelCounts;
 use crate::vm::{build_queues, Step, Vm};
 use crate::workload::Scenario;
 
@@ -111,6 +112,12 @@ pub struct Outcome {
     /// The longest a halted CPU sat beside a published surplus with no probe
     /// outstanding — see [`crate::vm::Vm::probe_gap_ns`].
     pub probe_gap_ns: u64,
+    /// Invariant I15's measurement: the most rounds a CPU stayed idle beside
+    /// stealable work, and the most a ready task waited to run.
+    pub wc_rounds: u64,
+    pub rounds_to_run: u64,
+    /// What the run did at each level of its machine's cache tree.
+    pub levels: LevelCounts,
     /// What each CPU's passes cost, as the core's `feature = "check"` recorder
     /// measured them — the on-target instrument, driven here by the scenario's
     /// modelled `pass_cost_ns`.
@@ -311,6 +318,9 @@ fn outcome_of(scenario: &'static str, vm: &Vm<'_>, choices: &ChoiceStream) -> Ou
         never_ran: vm.spawned.difference(&vm.ran).count(),
         idle_wakes: vm.idle_wakes.clone(),
         probe_gap_ns: vm.probe_gap_ns,
+        wc_rounds: vm.wc_rounds,
+        rounds_to_run: vm.rounds_to_run,
+        levels: vm.levels.clone(),
         pass_costs: (0..vm.handles.len())
             .map(|cpu| {
                 let cpu = CpuId(cpu as u32);

kernel/sim/src/hw_impl.rs

@@ -33,6 +33,9 @@ pub struct SimHwState {
     pub released: Vec<(TaskKey, TaskAccounting)>,
     pub trace: Vec<TraceEvent>,
     pub kicks: u64,
+    /// Each kick's target since the VM last drained it, which it does once a
+    /// step to say what level each one crossed.
+    pub kicked: Vec<CpuId>,
     pub switches: u64,
     /// Which CPU's pass is running. The core has no `cpu_id()` by design, so
     /// the driver is what knows where a timer or a halt lands.
@@ -70,6 +73,7 @@ impl SimHw {
                 released: Vec::new(),
                 trace: Vec::new(),
                 kicks: 0,
+                kicked: Vec::new(),
                 switches: 0,
                 current: None,
                 pass_cost_ns: 0,
@@ -118,6 +122,7 @@ impl Kicker for SimHw {
         self.with(|s| {
             s.pending_ipi[target.0 as usize] += 1;
             s.kicks += 1;
+            s.kicked.push(target);
         });
     }
 }

kernel/sim/src/invariants.rs

@@ -18,10 +18,10 @@ use kernel::sched::fair::{MAX_VRUNTIME_LAG_NS, QUANTUM_NS};
 use kernel::sched::invariants::{residents, Container};
 use kernel::sched::task::{TaskKey, TaskState};
 
-use kernel::sched::cpu::{DYING_AGE_NS, DYING_CHUNK_NS};
+use kernel::sched::cpu::{Balance, DYING_AGE_NS, DYING_CHUNK_NS};
 
 use crate::latency::ReadyCause;
-use crate::vm::{FairEpoch, Vm, IPI_LATENCY_NS, RUN_CHUNK_NS, UNWIND_NS};
+use crate::vm::{FairEpoch, Vm, IPI_LATENCY_NS, PROBE_WORTH_IT, RUN_CHUNK_NS, UNWIND_NS};
 
 /// How long a CPU may keep running a normal task while an RT task is ready on
 /// it (invariant I4): the interrupt's own delivery bound, plus the preempt-off
@@ -127,6 +127,7 @@ pub fn check_all(vm: &mut Vm<'_>) {
     check_share_refcounts(vm);
     check_address_spaces(vm);
     check_boost_windows(vm);
+    check_work_conservation(vm);
     note_run_latency(vm);
 }
 
@@ -140,7 +141,7 @@ pub fn check_all(vm: &mut Vm<'_>) {
 /// share's refcount, and it is exactly a thread whose competition is changing,
 /// which is the one thing I13 declines to measure across.
 fn runnable_now(vm: &Vm<'_>) -> (Vec<u32>, Vec<u32>, BTreeSet<TaskKey>) {
-    let cpus = vm.scenario.cpus;
+    let cpus = vm.scenario.cpus();
     let mut counted = vec![0u32; vm.procs.len()];
     let mut per_cpu = vec![0u32; vm.procs.len() * cpus];
     let mut threads = BTreeSet::new();
@@ -166,7 +167,7 @@ fn runnable_now(vm: &Vm<'_>) -> (Vec<u32>, Vec<u32>, BTreeSet<TaskKey>) {
 /// and I6 both need it and it is the more expensive half of either.
 fn runnable_per_process(vm: &Vm<'_>) -> Vec<u32> {
     let mut counted = vec![0u32; vm.procs.len()];
-    for cpu in 0..vm.scenario.cpus {
+    for cpu in 0..vm.scenario.cpus() {
         for (key, container) in residents(&vm.cpus[cpu]) {
             if !matches!(
                 container,
@@ -192,7 +193,7 @@ fn check_single_ownership(vm: &mut Vm<'_>) {
     let mut seen: BTreeMap<TaskKey, (usize, Container)> = BTreeMap::new();
     let mut problems = Vec::new();
 
-    for cpu in 0..vm.scenario.cpus {
+    for cpu in 0..vm.scenario.cpus() {
         for (key, container) in residents(&vm.cpus[cpu]) {
             if let Some((other, other_container)) = seen.insert(key, (cpu, container)) {
                 problems.push(format!(
@@ -257,7 +258,7 @@ fn check_single_ownership(vm: &mut Vm<'_>) {
 fn check_sleeping_cpus(vm: &mut Vm<'_>) {
     let mut problems = Vec::new();
     let (halted, pending) = vm.hw.with(|s| (s.halted.clone(), s.pending_ipi.clone()));
-    for cpu in 0..vm.scenario.cpus {
+    for cpu in 0..vm.scenario.cpus() {
         // The stopped CPU is the modelled fault, not a protocol break: it is
         // *given* work it never takes, which is the whole of what the scenario
         // stages. Every other CPU still answers for itself.
@@ -304,17 +305,14 @@ fn check_sleeping_cpus(vm: &mut Vm<'_>) {
 fn check_retires(vm: &mut Vm<'_>) {
     let mut problems = Vec::new();
 
-    let migrated: Vec<(TaskKey, u32)> = vm.hw.with(|s| {
-        let fresh = s.trace[vm.trace_cursor..]
-            .iter()
-            .filter_map(|ev| match ev.kind {
-                kernel::sched::hw::TraceKind::Migrate { task, to, .. } => Some((task, to.0)),
-                _ => None,
-            })
-            .collect();
-        vm.trace_cursor = s.trace.len();
-        fresh
-    });
+    let migrated: Vec<(TaskKey, u32)> = vm
+        .step_trace
+        .iter()
+        .filter_map(|ev| match ev.kind {
+            kernel::sched::hw::TraceKind::Migrate { task, to, .. } => Some((task, to.0)),
+            _ => None,
+        })
+        .collect();
     // Counted on the same walk that judges them: the policy suite's wakeup-storm
     // case claims a machine drains in parallel, and that claim is only about the
     // balance path if the balance path moved something.
@@ -409,7 +407,7 @@ fn retire_elapsed(vm: &Vm<'_>, key: TaskKey) -> u64 {
 /// thing the CPU owes. Delegated to the core's own checker, which
 /// is the same code a kernel `feature="check"` build runs.
 fn check_timers(vm: &mut Vm<'_>) {
-    for cpu in 0..vm.scenario.cpus {
+    for cpu in 0..vm.scenario.cpus() {
         let armed = vm.cpus[cpu].armed();
         let quantum = vm.cpus[cpu].running().map(|_| vm.cpus[cpu].quantum_end());
         let deadline = vm.cpus[cpu].parked().filter_map(|p| p.deadline()).min();
@@ -435,7 +433,7 @@ fn check_timers(vm: &mut Vm<'_>) {
 fn check_rt_latency(vm: &mut Vm<'_>) {
     let bound = rt_latency_bound(vm.max_kernel_section());
     let mut problems = Vec::new();
-    for cpu in 0..vm.scenario.cpus {
+    for cpu in 0..vm.scenario.cpus() {
         // `serves_rt_band` and not `is_rt`, for the reason that method's own doc
         // gives: a killed thread that holds the RT right is unwinding, not doing
         // real-time work, so a real-time sibling waiting behind it **is** being
@@ -477,7 +475,7 @@ fn check_rt_latency(vm: &mut Vm<'_>) {
 fn note_rt_service(vm: &mut Vm<'_>) -> bool {
     let mut rt = Vec::new();
     let mut occupied = false;
-    for cpu in 0..vm.scenario.cpus {
+    for cpu in 0..vm.scenario.cpus() {
         let sched = &vm.cpus[cpu];
         if let Some(task) = sched.running() {
             if task.rt().is_rt() {
@@ -572,7 +570,7 @@ fn check_fairness(vm: &mut Vm<'_>, rt_present: bool) {
     vm.fair_window_open = false;
     vm.thread_window_open = false;
     let (runnable, per_cpu, live_threads) = runnable_now(vm);
-    let saturated = (0..vm.scenario.cpus).all(|cpu| vm.cpus[cpu].running().is_some());
+    let saturated = (0..vm.scenario.cpus()).all(|cpu| vm.cpus[cpu].running().is_some());
     let mut members: Vec<usize> = if rt_present || !saturated {
         Vec::new()
     } else {
@@ -588,7 +586,7 @@ fn check_fairness(vm: &mut Vm<'_>, rt_present: bool) {
     // can still fill the machine.
     if members
         .iter()
-        .any(|&p| runnable[p] as usize * members.len() < vm.scenario.cpus)
+        .any(|&p| runnable[p] as usize * members.len() < vm.scenario.cpus())
     {
         members.clear();
     }
@@ -603,14 +601,14 @@ fn check_fairness(vm: &mut Vm<'_>, rt_present: bool) {
     // them sharing a CPU with a solo thread that is catching up, which is I5's
     // entitlement being honoured and not a sibling being cheated.
     let balanced = members.iter().all(|&process| {
-        let row = &per_cpu[process * vm.scenario.cpus..(process + 1) * vm.scenario.cpus];
+        let row = &per_cpu[process * vm.scenario.cpus()..(process + 1) * vm.scenario.cpus()];
         row.iter().min() == row.iter().max()
     });
     let threads: u32 = members.iter().map(|&p| runnable[p]).sum();
     // What one CPU's fair band holds. Read off CPU 0 rather than divided out of
     // the machine-wide total, so it is the quantity itself and not an average
     // that happens to equal it; `balanced` is what makes every CPU's the same.
-    let rivals: u32 = members.iter().map(|&p| per_cpu[p * vm.scenario.cpus]).sum();
+    let rivals: u32 = members.iter().map(|&p| per_cpu[p * vm.scenario.cpus()]).sum();
 
     if members != vm.fair_epoch.members {
         vm.fair_epoch = FairEpoch {
@@ -852,7 +850,7 @@ fn check_thread_service(vm: &mut Vm<'_>, members: &[usize]) {
                 // Which run queue each thread is in, because that is the first
                 // question a per-thread spread raises and the answer is one
                 // scan on a path that has already failed.
-                let at = (0..vm.scenario.cpus)
+                let at = (0..vm.scenario.cpus())
                     .find(|&cpu| residents(&vm.cpus[cpu]).any(|(k, _)| k == *key))
                     .map_or_else(|| "?".to_string(), |cpu| cpu.to_string());
                 Some(format!("{}@cpu{at}={served}", key.0))
@@ -871,6 +869,99 @@ fn check_thread_service(vm: &mut Vm<'_>, members: &[usize]) {
     }
 }
 
+/// I15: **work conservation.** No CPU spends more than
+/// [`work_conservation_bound`] protocol rounds ([`crate::vm::Vm::round`]) idle
+/// while another holds a task a probe would be handed — a fair band at least
+/// [`PROBE_WORTH_IT`] deep.
+///
+/// The bounded form of the rule rather than a predicate over one step, and the
+/// reason is what a step lets the simulator see. Its safety form — no CPU parks
+/// unprobed beside stealable work that nobody rang it for — turns on whether a
+/// victim's push rang a parked CPU, and a ring on a doorbell whose edge is
+/// already raised changes nothing outside the core: it reads exactly as no ring.
+/// What is visible is the outcome, an idle CPU and a queue that stays stealable,
+/// and a protocol that leaves work unannounced or refuses it keeps that outcome
+/// standing round after round. So the bound is on the outcome, and every
+/// failure the safety form names fails it: an idle CPU nobody rings, one that is
+/// rung and declines to probe, a probe that is never answered.
+///
+/// **Nothing pauses the count.** A ring or a task on its way to the idle CPU
+/// lands after an interrupt's latency, a matter of time, which the rest of the
+/// machine could fill with rounds that take none; a round waits on every CPU an
+/// IPI is on its way to ([`crate::vm::Vm::round`]), so an answer in flight costs
+/// at most the round it was sent in. A CPU that is rung and declines to probe,
+/// or whose probe is never answered, accrues every round it stays idle however
+/// often it is rung.
+///
+/// Rounds and not nanoseconds, because the clock measures the explorer as much
+/// as the protocol; and a stopped CPU is neither idle nor a victim, its numbers
+/// being the modelled fault `stopped_cpu` measures.
+fn check_work_conservation(vm: &mut Vm<'_>) {
+    let cpus = vm.scenario.cpus();
+    let victim = (0..cpus).find(|&cpu| {
+        !vm.is_stopped(cpu) && vm.cpus[cpu].rq().fair_len() as u32 >= PROBE_WORTH_IT
+    });
+    let bound = work_conservation_bound(vm.scenario.balance, cpus).unwrap_or(u64::MAX);
+    let completed = vm.round - std::mem::replace(&mut vm.wc_round_seen, vm.round);
+    let mut problems = Vec::new();
+    for cpu in 0..cpus {
+        let idle = !vm.is_stopped(cpu) && vm.cpus[cpu].running().is_none();
+        let Some(victim) = victim.filter(|_| idle) else {
+            vm.wc_idle[cpu] = None;
+            continue;
+        };
+        let rounds = vm.wc_idle[cpu].get_or_insert(0);
+        *rounds += completed;
+        let rounds = *rounds;
+        vm.wc_rounds = vm.wc_rounds.max(rounds);
+        if rounds > bound {
+            problems.push(format!(
+                "I15: cpu{cpu} has been idle for {rounds} rounds while cpu{victim} holds {} \
+                 queued fair task(s) (bound {bound} rounds)",
+                vm.cpus[victim].rq().fair_len(),
+            ));
+        }
+    }
+    for problem in problems {
+        vm.violate(problem);
+    }
+}
+
+/// How many rounds a CPU may spend idle beside stealable work: `cpus + 2` under
+/// [`Balance::PushOnSurplus`], and no bound under a policy that claims none —
+/// the pull alone, no balance path and the declined re-arm, the controls
+/// `sim/tests/policy.rs` prices the push against.
+///
+/// **Derived for one victim.** An idle CPU `p` beside a victim `v`'s stealable
+/// work counts:
+///
+/// 1. `cpus` rounds until `p` is rung: the round open when `p` began counting,
+///    whose pass of `v` may have come before it, and `cpus − 1` more. `v` runs
+///    a task, so it is awake when every round opens and passes in each; every
+///    such pass that leaves it stealable rings the first sleeping CPU at or
+///    after its cursor and moves the cursor past it, so `p` is reached within
+///    `cpus − 1` of them. A `p` the cursor passes because it is awake owes the
+///    next round its pass, which probes as term 2's does.
+/// 2. One round for `p`'s idle pass. A ring still in flight when the next round
+///    opens holds that round open until `p` has passed, so `p` has probed `v`,
+///    the only CPU publishing a surplus worth a probe, before it completes.
+/// 3. One round for `v`'s answer: `v` passes in the round after the probe was
+///    posted, and answers. The answer hands over a task unless that leaves `v`
+///    with none to spare — and then nothing is stealable and the count is over
+///    — and its post leaves `p` awake or rung, so the next round waits on the
+///    pass in which `p` takes it and runs it.
+///
+/// With several victims the derivation does not hold: each walks its own
+/// cursor, and a probe sent to one that runs dry before it answers leaves `p`
+/// to wait for a ring again — the flat push's weakness, recorded in
+/// `issues/the-push-bounds-an-idle-cpus-wait-only-for-one-victim.md`.
+fn work_conservation_bound(balance: Balance, cpus: usize) -> Option<u64> {
+    match balance {
+        Balance::PushOnSurplus { .. } => Some(cpus as u64 + 2),
+        Balance::None | Balance::Pull | Balance::PullWithRearm { .. } => None,
+    }
+}
+
 /// The measured policy suite's instrument: how long each task waited between
 /// being owed a dispatch and getting one (`crate::latency`, `sim/tests/policy.rs`).
 ///
@@ -895,7 +986,7 @@ fn check_thread_service(vm: &mut Vm<'_>, members: &[usize]) {
 fn note_run_latency(vm: &mut Vm<'_>) {
     let now = vm.clock;
     let mut seen: BTreeMap<TaskKey, Container> = BTreeMap::new();
-    for cpu in 0..vm.scenario.cpus {
+    for cpu in 0..vm.scenario.cpus() {
         for (key, container) in residents(&vm.cpus[cpu]) {
             seen.insert(key, container);
         }
@@ -925,10 +1016,11 @@ fn note_run_latency(vm: &mut Vm<'_>) {
                         | Container::Zombie,
                     ) => ReadyCause::Fresh,
                 };
-                vm.awaiting.insert(key, (now, cause));
+                vm.awaiting.insert(key, (now, cause, vm.round));
             }
         } else if container == Container::Running {
-            if let Some((since, cause)) = vm.awaiting.remove(&key) {
+            if let Some((since, cause, round)) = vm.awaiting.remove(&key) {
+                vm.rounds_to_run = vm.rounds_to_run.max(vm.round - round);
                 dispatched.push((key, cause, now.since(since)));
             }
         } else {
@@ -1091,7 +1183,7 @@ pub fn check_final(vm: &mut Vm<'_>) {
             stuck.join(", "),
         ));
     }
-    for cpu in 0..vm.scenario.cpus {
+    for cpu in 0..vm.scenario.cpus() {
         if vm.is_stopped(cpu) {
             continue;
         }

kernel/sim/src/lib.rs

@@ -33,5 +33,6 @@ pub mod payload;
 pub mod scenarios;
 pub mod shrink;
 pub mod sweep;
+pub mod topology;
 pub mod vm;
 pub mod workload;