LAN stage D and B: netstack wakes only on a change, and the I219 takes a 1,000-descriptor ring at a fixed 131 µs throttle

issues/closing-a-pipe-end-ends-every-holders-polls-on-it.md

@@ -0,0 +1,24 @@
+---
+status: open
+kind: defect
+opened: 2026-10-11
+---
+
+# Closing a pipe end ends every holder's polls on it
+
+`ops::close` (`kernel/src/object/ops.rs`) cancels every poll on the watches of
+the object the closed handle names: a pipe end's data watches and its leaving
+watch, and those of a connection, an acceptor, a file and the other objects
+`close_ends_polls` lists. `Watch::cancel_rings` (`kernel/pure/sched/watch.rs`)
+takes every ring registered on the watch, whoever registered it. So when two
+handles name one pipe end — a duplicate, or a handle moved to another process
+that kept one — closing either answers the other's poll on its live handle
+with `NotFound`, as though that handle had been closed. Its holder reads a
+watch it still needs as over.
+
+Found by reading, in review of pull request #861, which put the pipe end's
+leaving watch in that same loop. No test or machine has shown it.
+
+**Exit**: a close ends only the polls made through the handle it closes, and a
+guest test that holds a duplicate of a pipe end, watches it, and closes the
+other handle reads no answer on it.

issues/netstacks-latency-under-a-receive-batch-over-64-frames-is-unmeasured.md

@@ -0,0 +1,35 @@
+---
+status: open
+kind: tooling
+opened: 2026-10-12
+---
+
+# netstack's latency under a receive batch of more than 64 frames is unmeasured
+
+`Node::receive` takes at most the card's ring in one batch (`Card::receive`,
+`system/netstack/src/card.rs`): `toyos_i219::RX_RING - 1`, 999 frames, on the
+I219, and the queue's 256 on virtio. Before pull request #861 it took 64. Every
+frame of a batch reaches the stack under one `now`, read before the batch
+(`system/netstack/src/main.rs`'s loop), and the deadlines that are due, the
+transmit opportunity and the clients' answers run only after the batch's last
+frame. So a backlog that was served as several 64-frame passes, with those
+three between them, is now one pass: a frame late in it is stamped with the
+batch's start, and a deadline due during it fires after it.
+
+Nothing moves while a pass brings 64 frames or fewer, and nothing has read a
+pass that brings more. The T14's LAN round trip was read only under one
+gigabit download of full-sized frames, at about 12 frames a pass (round 4 of
+pull request #861: 59.7 passes per MB, and some 700 full-sized frames carry a
+MB; derived). A pass brings more than 64 when netstack is about 0.79 ms behind
+full-sized frames at a gigabit (64 frames of 12.3 µs on the wire, derived), or
+under minimum-sized frames, of which a gigabit carries about 195 in one
+131 µs throttle interval (672 ns a frame on the wire, derived).
+
+**Exit**: a T14 reading, in one session, of the LAN round trip and of how late
+netstack fires a due deadline, under a load that brings more than 64 frames a
+pass, with the ring's budget and with a budget of 64, recorded in the commit
+that closes this. A gap between the two is investigated where it shows, and is
+no verdict on the budget by itself.
+
+**Owner**: whoever holds `issues/the-lan-is-not-yet-production-grade.md`, whose
+stage 2 sets the ring sizes and the interrupt moderation this batch follows.

issues/netstacks-receiver-round-trip-reads-no-finer-than-a-millisecond.md

@@ -0,0 +1,22 @@
+---
+status: open
+kind: defect
+opened: 2026-10-11
+---
+
+# netstack's receiver round trip reads no finer than a millisecond
+
+A downloading stream's round trip is the receiver's own estimate
+(`system/netstack/shard/tcp/src/rx.rs`): the age of an echoed TSval. TSval
+ticks once a millisecond, so an echo's age is a whole number of milliseconds
+and never less than one tick. A path whose round trip is under a millisecond,
+a LAN's, reads as at least 1 ms.
+
+What that moves is the receive buffer's growth, twice what the user read per
+round trip (`rx.rs`): a 200 µs path that reads 1 ms grows its buffer to five
+times what it needs. That figure is derived from the tick and the growth rule.
+No machine has measured a sub-millisecond path.
+
+**Exit**: a stream on a sub-millisecond path grows its receive buffer to twice
+what the user reads in the path's own round trip, read by a host test that
+delivers echoes on such a path.

issues/the-lan-is-not-yet-production-grade.md

@@ -40,7 +40,7 @@ why, and the boot still leases.
   `issues/no-machine-has-read-an-intel-nic-across-a-link-change.md` waits on it.
 - **Throughput** is measured against Ubuntu on the same machine and cable, with
   line rate (about 940 Mb/s of TCP) as the target: interrupt moderation (a
-  fixed 128 µs throttle today, not adaptive), ring sizes, checksum and
+  fixed throttle today, not adaptive), ring sizes, checksum and
   segmentation offload.
 - **The part is handed back** in the state the next operating system and the
   management firmware that shares it expect. Ubuntu has had link after every

kernel/src/inbox/mod.rs

@@ -536,9 +536,10 @@ fn arm(
     let ready = readiness_of(object, flags).any();
     let read = if flags.readable() { ops::read_watch(object) } else { None };
     let write = if flags.writable() { ops::write_watch(object) } else { None };
-    // A pipe end's own watch is the one its other end's last holder posts;
-    // an object with no other end is refused the question, whatever else the
-    // watch asks. A direction asked above has taken that watch already.
+    // A pipe end's leaving watch is posted by its other end's last holder
+    // alone; an object with no other end is refused the question, whatever
+    // else the watch asks. A direction asked above needs no second entry: that
+    // last holder posts the end's own data watch too.
     let own = match flags.other_end_gone().then(|| ops::pipe_end_watch(object)) {
         Some(None) => return Err(SyscallError::NotSupported),
         Some(Some(_)) if read.is_some() || write.is_some() => None,

kernel/src/object/ops.rs

@@ -188,7 +188,7 @@ pub fn close(
     // Only a watch this handle's object ends is answered: one it shares (every `Console` and a
     // `Device(Keyboard)` name the keyboard's) cannot cancel another's poll.
     if close_ends_polls(&object) {
-        for watch in [read_watch(&object), write_watch(&object)].into_iter().flatten() {
+        for watch in [read_watch(&object), write_watch(&object), pipe_end_watch(&object)].into_iter().flatten() {
             watch.cancel_polls();
         }
     }
@@ -315,12 +315,12 @@ pub fn write_watch(object: &KObjectRef) -> Option<WatchRef> {
     }
 }
 
-/// The watch of a pipe end, which its other end's last holder posts as it
-/// lets go; `None` for an object that is no pipe end.
+/// A pipe end's leaving watch, which only its other end's last holder posts as
+/// it lets go; `None` for an object that is no pipe end.
 pub fn pipe_end_watch(object: &KObjectRef) -> Option<WatchRef> {
     match object {
-        KObjectRef::PipeRead(_) => read_watch(object),
-        KObjectRef::PipeWrite(_) => write_watch(object),
+        KObjectRef::PipeRead(r) => pipe::left_watch(r.id(), End::Read).map(WatchRef::Shared),
+        KObjectRef::PipeWrite(w) => pipe::left_watch(w.id(), End::Write).map(WatchRef::Shared),
         KObjectRef::Connection(_) | KObjectRef::Acceptor(_) | KObjectRef::Process(_)
         | KObjectRef::Console(_) | KObjectRef::Device(_) | KObjectRef::SysCap(_)
         | KObjectRef::File(_) | KObjectRef::Inbox(_) | KObjectRef::Connector(_)

kernel/src/pipe.rs

@@ -117,6 +117,10 @@ struct Pipe {
     /// poll registration can clone one out from under the table lock and hold it across its own park.
     read_watch: Arc<Watch>,
     write_watch: Arc<Watch>,
+    /// Each end's leaving watch, by [`End`]: an `OTHER_END_GONE` poll on that
+    /// end sits on it, and only the other end's last holder posts it — a byte
+    /// moved through the pipe wakes no poll that asked about a leaving.
+    left: [Arc<Watch>; 2],
     /// Set when a write should hand the next reader transient RT priority — covers readers already runnable, which the wake-time boost misses.
     rt_boost_pending: bool,
 }
@@ -132,6 +136,7 @@ impl Pipe {
             holders: Holders::new(),
             read_watch: Arc::new(Watch::new()),
             write_watch: Arc::new(Watch::new()),
+            left: [Arc::new(Watch::new()), Arc::new(Watch::new())],
             rt_boost_pending: false,
         }
     }
@@ -283,23 +288,33 @@ pub fn set_rt_boost_pending(pipe_id: PipeId) {
     });
 }
 
-/// One reference to `end` lets go; its end's last posts the other end's watch.
+/// One reference to `end` lets go; its end's last posts the other end's watch
+/// and its leaving watch.
 fn release(pipe_id: PipeId, end: End) {
-    let released = with_pipes_mut(|pipes| {
+    let (released, left) = with_pipes_mut(|pipes| {
         let pipe = pipes.get_mut(pipe_id).expect("release: pipe not found");
         let released = pipe.holders.release(end);
         pipe.publish_ends();
+        let left = pipe.left[slot(end.other())].clone();
         if released == Released::PipeGone {
             let pipe = pipes.remove(pipe_id).unwrap();
             free_pipe(pipe);
         }
-        released
+        (released, left)
     });
     match (released, end) {
-        (Released::Held | Released::PipeGone, _) => {}
+        (Released::Held | Released::PipeGone, _) => return,
         (Released::EndGone, End::Read) => crate::scheduler::wake_pipe_writers(pipe_id),
         (Released::EndGone, End::Write) => crate::scheduler::wake_pipe_readers(pipe_id),
     }
+    left.post();
+}
+
+fn slot(end: End) -> usize {
+    match end {
+        End::Read => 0,
+        End::Write => 1,
+    }
 }
 
 fn free_pipe(pipe: Pipe) {
@@ -315,3 +330,9 @@ pub fn read_watch(pipe_id: PipeId) -> Option<Arc<Watch>> {
 pub fn write_watch(pipe_id: PipeId) -> Option<Arc<Watch>> {
     with_pipes(|pipes| pipes.get(pipe_id).map(|p| p.write_watch.clone()))
 }
+
+/// `end`'s leaving watch, the same way: posted once, by the other end's last
+/// holder.
+pub fn left_watch(pipe_id: PipeId, end: End) -> Option<Arc<Watch>> {
+    with_pipes(|pipes| pipes.get(pipe_id).map(|p| p.left[slot(end)].clone()))
+}

system/netstack/i219/src/lib.rs

@@ -148,25 +148,39 @@ pub trait Interrupts {
     fn taken(&self) -> Result<u32, Self::Refused>;
 }
 
-/// How many receive descriptors the ring holds. §7.1.8: `RDLEN` "must be a
-/// multiple of 128", so the count is a multiple of eight.
-pub const RX_RING: usize = 256;
+/// How much memory the driver asks to be granted, in one region: one 2 MiB
+/// span, which is what the kernel maps a grant of any smaller size in, so
+/// every byte of it is the receive ring's that the transmit ring does not
+/// need.
+pub const GRANT_BYTES: u64 = 2 << 20;
+/// How many receive descriptors the ring holds: the most whose descriptors
+/// and buffers fit the grant beside the transmit ring's. §7.1.8: `RDLEN`
+/// "must be a multiple of 128", so the count is a multiple of eight.
+pub const RX_RING: usize = 1000;
 /// How many transmit descriptors the ring holds. A frame is handed to the
 /// device the moment it is filled and reclaimed when room is next asked for,
 /// so what this bounds is how many may be in flight at once: one fewer than
 /// this, [`I219::tx_room`]'s most.
 pub const TX_RING: usize = 16;
-/// `ITR` (§10.2.4.2): the least interval between two of the function's
-/// interrupts, in 256 ns units — 128 µs. It runs from the last interrupt, so a
-/// frame after a quiet spell that long interrupts at once, and a gigabit
-/// stream's full-sized frames, one each 12.3 µs, wake their reader about ten at
-/// a time.
-const THROTTLE: u32 = 500;
+/// Free receive descriptors at which the part reports itself short:
+/// `RCTL.RDMTS = 00b`, "1/2" of `RDLEN` (§10.2.5.1), raising `RXDMT0`.
+const RX_SHORT: usize = RX_RING / 2;
+/// Receive buffers given back and not yet handed to the part at most: the
+/// tail moves once a batch ([`I219::rx_flush`]), and sooner once this many
+/// wait. A quarter of [`RX_SHORT`], so what a batch holds back never brings
+/// the part to its report alone.
+pub const HELD_MAX: usize = RX_SHORT / 4;
 /// Bytes per receive buffer — `RCTL.BSIZE = 00b` with `BSEX` clear (§10.2.5.1).
 pub const RX_BUF_BYTES: usize = 2048;
 /// Bytes per transmit buffer, and therefore the longest frame [`I219::tx_reserve`]
 /// takes.
 pub const TX_BUF_BYTES: usize = 2048;
+/// `ITR` (§10.2.4.2): the least interval between two of the function's
+/// interrupts, in 256 ns units — 131 µs. It runs from the last interrupt, so a
+/// frame after a quiet spell that long interrupts at once, and a gigabit
+/// stream's full-sized frames, one each 12.3 µs, wake their reader about ten at
+/// a time.
+const THROTTLE: u32 = 512;
 
 const _: () = {
     // §7.1.8 and §7.2.4: both ring lengths are programmed in bytes and "must be
@@ -181,20 +195,30 @@ const _: () = {
 
 /// The grant's layout. One grant, because every byte of it is this process's
 /// own: what a wrong offset here corrupts is this driver's memory and nobody
-/// else's.
+/// else's. Each buffer starts on a multiple of its own size.
 const OFF_RX_RING: usize = 0x0000;
 const OFF_TX_RING: usize = OFF_RX_RING + RX_RING * rx_desc::BYTES;
-const OFF_RX_BUFS: usize = 0x2000;
+const OFF_RX_BUFS: usize = layout(RX_RING).0;
 const OFF_TX_BUFS: usize = OFF_RX_BUFS + RX_RING * RX_BUF_BYTES;
-/// How much memory the driver needs granted, in one region.
-pub const GRANT_BYTES: u64 = (OFF_TX_BUFS + TX_RING * TX_BUF_BYTES) as u64;
+
+/// Where the receive buffers start, and where the grant's last byte ends, for
+/// a receive ring of `rx` descriptors.
+const fn layout(rx: usize) -> (usize, usize) {
+    let rings = rx * rx_desc::BYTES + TX_RING * tx_desc::BYTES;
+    let rx_bufs = rings.div_ceil(RX_BUF_BYTES) * RX_BUF_BYTES;
+    (rx_bufs, rx_bufs + rx * RX_BUF_BYTES + TX_RING * TX_BUF_BYTES)
+}
 
 const _: () = {
     assert!(OFF_TX_RING + TX_RING * tx_desc::BYTES <= OFF_RX_BUFS);
+    // The grant holds this ring and not the next multiple of eight.
+    assert!(layout(RX_RING).1 <= GRANT_BYTES as usize);
+    assert!(layout(RX_RING + 8).1 > GRANT_BYTES as usize);
     // §7.1.8 and §7.2.4: the ring base "is aligned on a 16-byte boundary" and
     // "hardware ignores the lower 4 bits", so a ring at an unaligned offset
     // would be programmed somewhere else entirely.
     assert!(OFF_RX_RING.is_multiple_of(16) && OFF_TX_RING.is_multiple_of(16));
+    assert!(HELD_MAX > 0);
 };
 
 /// Why the function was not brought up. Each keeps its own word: a caller asks
@@ -765,6 +789,8 @@ pub struct I219<R: Registers, C, D, I> {
     /// receive ring keeps in software's hands (§7.1.8: the queue is empty when
     /// head equals tail).
     rx_tail: usize,
+    /// Buffers given back since the tail last moved.
+    rx_held: usize,
     /// Next transmit descriptor to hand out, which is also `TDT`.
     tx_next: usize,
     /// Next transmit descriptor to reclaim.
@@ -913,6 +939,7 @@ impl<R: Registers, C: Clock, D: DmaBuffers, I: Interrupts> I219<R, C, D, I> {
             link_up_at: None,
             rx_next: 0,
             rx_tail: RX_RING - 1,
+            rx_held: 0,
             tx_next: 0,
             tx_clean: 0,
             tx_wake: false,
@@ -1253,19 +1280,20 @@ impl<R: Registers, C: Clock, D: DmaBuffers, I: Interrupts> I219<R, C, D, I> {
     }
 
     /// Give a frame's buffer back to the device, its bytes having been read.
+    /// It reaches the device at the next [`Self::rx_flush`], or here once
+    /// [`HELD_MAX`] buffers wait.
     pub fn rx_done(&mut self, frame: Frame) {
         self.give_back(frame.index);
     }
 
-    /// Republish descriptor `index` and move the tail over every ready
-    /// descriptor from where it stands.
-    ///
-    /// **The tail moves over a run, never to an index.** §7.1.8's tail
-    /// "identifies the location beyond the last descriptor hardware can
-    /// process", so writing it with the descriptor that came back last would
-    /// hand the device every descriptor between — including one whose bytes
-    /// their reader still holds — and, for a buffer returned before an older
-    /// one, walk `RDT` backwards over the rest of the ring.
+    /// Hand the device every buffer given back since the tail last moved: the
+    /// end of a batch, and one `RDT` write for all of it.
+    pub fn rx_flush(&mut self) {
+        self.move_tail();
+    }
+
+    /// Republish descriptor `index`, and move the tail once [`HELD_MAX`]
+    /// buffers wait for it.
     ///
     /// **Ready is read out of the descriptor, so there is no second record of
     /// it to disagree.** [`publish_rx`] zeroes the status word, and a
@@ -1274,6 +1302,22 @@ impl<R: Registers, C: Clock, D: DmaBuffers, I: Interrupts> I219<R, C, D, I> {
     /// touches neither until the tail passes it.
     fn give_back(&mut self, index: usize) {
         self.publish_rx(index);
+        self.rx_held += 1;
+        if self.rx_held >= HELD_MAX {
+            self.move_tail();
+        }
+    }
+
+    /// Move the tail over every ready descriptor from where it stands.
+    ///
+    /// **The tail moves over a run, never to an index.** §7.1.8's tail
+    /// "identifies the location beyond the last descriptor hardware can
+    /// process", so writing it with the descriptor that came back last would
+    /// hand the device every descriptor between — including one whose bytes
+    /// their reader still holds — and, for a buffer returned before an older
+    /// one, walk `RDT` backwards over the rest of the ring.
+    fn move_tail(&mut self) {
+        self.rx_held = 0;
         let mut tail = self.rx_tail;
         loop {
             let next = (tail + 1) % RX_RING;