diff --git a/issues/a-stream-its-peer-reset-refuses-the-option-requests-a-host-answers.md b/issues/a-stream-its-peer-reset-refuses-the-option-requests-a-host-answers.md index 91c4fa4772f..2bc2bad0317 100644 --- a/issues/a-stream-its-peer-reset-refuses-the-option-requests-a-host-answers.md +++ b/issues/a-stream-its-peer-reset-refuses-the-option-requests-a-host-answers.md @@ -9,7 +9,8 @@ opened: 2026-10-08 The node lets a stream go once it holds neither of its pipe ends (`userland/netstack/node/src/streams.rs`), and netstack's id for it goes with it (`userland/netstack/src/serve.rs`, `Sockets::settle`). A peer's reset ends -both pipes at once, so a client that still holds the stream names an id +both pipes once the client's pipe has taken the text received before it, +so a client that still holds the stream names an id netstack no longer has, and `Sockets::set_option` answers it `ERR_NOT_CONNECTED`. diff --git a/issues/netstacks-virtio-receive-has-no-per-pass-budget.md b/issues/netstacks-virtio-receive-has-no-per-pass-budget.md new file mode 100644 index 00000000000..39c8e5d31de --- /dev/null +++ b/issues/netstacks-virtio-receive-has-no-per-pass-budget.md @@ -0,0 +1,31 @@ +--- +status: open +kind: defect +opened: 2026-10-10 +--- + +# netstack's virtio receive has no per-pass budget + +`VirtioNet::poll_rx` (`userland/netstack/src/virtio_net.rs`) answers a frame for +as long as the used ring holds one, and `Node::receive` +(`userland/netstack/node/src/lib.rs`) takes frames until the card says none is +left. A device that refills the ring as fast as it is read keeps one pass in +receive: the transmit opportunity, the deadlines and every stream's pass wait +for the ring to empty, and the whole batch is read at one `now`. The I219 is +bounded at `toyos_i219::RX_BUDGET` frames a pass. + +The I219's bound is not one to copy. A pass that stops at it leaves frames whose +interrupts the pass's `begin_pass` already took, and nothing in netstack's loop +(`userland/netstack/src/main.rs`) makes the next wait return at once, so they +wait for the next interrupt or deadline. Read from the code, not measured. + +The fix is one bound for every driver where the pass takes its frames, at +`Node::receive`, answering that it stopped at the bound so the loop's next wait +is zero; the I219's own budget is then deleted. That touches `toyos-i219`, which +the branch that found this (#828) was not briefed to change. + +**Exit condition**: a host test feeds `Node::receive` a source that never empties +and sees it stop at the bound and answer that a pass is owed; netstack's loop +waits zero after such a pass; `toyos_i219::RX_BUDGET` is gone. + +**Owner**: whoever holds `issues/toyos-has-its-own-network-stack.md`. diff --git a/issues/toyos-has-its-own-network-stack.md b/issues/toyos-has-its-own-network-stack.md index 33327657315..cadef878273 100644 --- a/issues/toyos-has-its-own-network-stack.md +++ b/issues/toyos-has-its-own-network-stack.md @@ -30,9 +30,9 @@ What the node does not yet meet: - A listener's handshakes in progress are [tcp]'s `limits::LISTEN_PENDING`, each held until `limits::SYNACK_GIVE_UP`, and a SYN past them is dropped: 256 SYNs a minute from addresses that never answer shut a port to every other peer for as long as they keep coming. [tcp] has no SYN cookie and lets no older handshake go for a newer one. Exit: a test in `toyos-net-shard/tcp/tests/` in which a peer's handshake completes while LS-12's flood runs. - The node counts a wake unspent until an accept arrives, and cannot tell an accept on its way from one that will never be sent: an owner that reads a wake and sends no accept is owed one wake fewer from then on, and the last connection waiting is not announced. That is the node's half of `issues/an-accept-that-never-reaches-netstack-strands-its-listener.md`. Exit: that issue's, by a client that cannot spend a wake without its accept arriving, or by an accept that waits in netd and needs no wake. - A stream its client can see no more, its reader gone or the peer's FIN read through and its client writing no more, is reset once its pipe has given up no byte for 100 s, and a peer address restarts that clock for at most 16 such streams at once (`OWNERLESS_PER_PEER`, `userland/netstack/node/src/streams.rs`); one past them has 100 s from the pass that found it so, until one of the 16 is done and it takes its place. What holds in this stage, from the code: per address, 16, an estimate no measurement set. There is no floor on the rate: each of the 16 is kept for as long as its pipe gives up one byte per 100 s, which for a full 2 MiB pipe is 2,097,152 x 100 s, over six years, and then for its tail in [tcp] (the line below on a client gone with nothing left in its pipe). The 16 is no bound across addresses: addresses are not counted, and where a connect's address is its client's choice, an accepted stream's is the peer's, which on the link answers for as many addresses as it likes. The bound across them is the node's places (`userland/netstack/node/src/places.rs`), of which every stream holds one until the node lets it go, and there is no second one: in `peers_at_more_addresses_than_there_are_places_hold_no_stream_past_them` (`userland/netstack/node/tests/listeners.rs`) peers at four addresses bring sixteen connections each to a node with three places, which holds two such streams at a time, each kept alive past its 100 s, and refuses every accept and connect past them. Exit: the T14 or a guest shows a peer holding one at a byte per 100 s, and the cut takes a floor on the rate, or the owner rules it needs none. -- A stream its peer reset is gone from the node at once, so for a client that still holds it `Node::set_nodelay` answers `false`: `issues/a-stream-its-peer-reset-refuses-the-option-requests-a-host-answers.md`, carried by the node as built. std's `nodelay()` asks netd nothing: it answers from the value its `TcpStream` holds (`sdk/std/sys/net/connection.rs`), and libc's `getsockopt` of `TCP_NODELAY` from its socket's entry (`userland/libc/src/socket.rs`). What the node carries is `set_nodelay` on a reset stream, refused where macOS refuses it under another kind and Linux answers. The node cannot hold the stream for the client: its two pipe ends are all it has of the client, letting both go is how the client learns of the reset, and with both gone nothing tells it when the client leaves, so a kept stream would be kept for ever. Exit, outside the node: that issue's guest test is green, and a `getsockopt` of `TCP_NODELAY` on a stream its peer reset is answered, by libc from a value its socket holds as std's does or by a pipe ABI that gives netd a handle whose other end's leaving the kernel reports. +- A stream its peer reset is gone from the node once its client's pipe has taken the text received before the reset, so for a client that still holds it `Node::set_nodelay` answers `false`: `issues/a-stream-its-peer-reset-refuses-the-option-requests-a-host-answers.md`, carried by the node as built. std's `nodelay()` asks netd nothing: it answers from the value its `TcpStream` holds (`sdk/std/sys/net/connection.rs`), and libc's `getsockopt` of `TCP_NODELAY` from its socket's entry (`userland/libc/src/socket.rs`). What the node carries is `set_nodelay` on a reset stream, refused where macOS refuses it under another kind and Linux answers. The node cannot hold the stream for the client: its two pipe ends are all it has of the client, letting both go is how the client learns of the reset, and with both gone nothing tells it when the client leaves, so a kept stream would be kept for ever. Exit, outside the node: that issue's guest test is green, and a `getsockopt` of `TCP_NODELAY` on a stream its peer reset is answered, by libc from a value its socket holds as std's does or by a pipe ABI that gives netd a handle whose other end's leaving the kernel reports. - How a stream ended reaches its client as the order its two pipes end in, which is one bit: the peer's FIN, or a failure. The node keeps a send pipe whose FIN is queued until its writer leaves and lets a failed stream's send pipe go first (`userland/netstack/node/src/streams.rs`), so std and libc read a FIN as a FIN and a failure as `ConnectionReset` or `ECONNRESET`, whichever failure it was: `issues/a-streams-failure-reaches-its-client-as-a-reset-whatever-it-was.md`. Exit: that issue's. -- Every frame and every deadline ends in a pass over every stream, every transmit opportunity walks every stream twice (once for the count each peer address keeps alive, once to pass those whose connect is not answered), and `Node::next_deadline` reads every stream: linear in streams per frame, as netstack's bridge on smoltcp was. For an idle stream a pass is one `recv_with`, two `status` and one read of its client's pipe, which in netstack is a system call: 1,000 system calls a frame with 1,000 idle streams. A pass's `recv_with` also re-files its connection's deadline and offers it to the transmit round, so one frame makes every idle stream eligible and the next opportunity serves each to learn it has nothing. [tcp] cannot say which connections a segment, a timer, an ICMP error or a failed next hop moved: it reports `drain_eligible` and `drain_gone`, which are the round's. The change: [tcp] marks a held connection in `for_conn`, `tick`, `icmp` and `end` and hands the marks out once (`drain_moved`); the node maps a connection to its stream, keeps its deadlines ordered, and passes only the streams [tcp] moved, the one a pipe's wake names, and those due. A pipe's end-gone answer (`Node::pipe_gone`) runs a pass of its own as well, beside the one `Sockets::bridge` runs for a wake's readiness answers, so k clients leaving in one wake cost k + 1 passes. Not measured at the move. The count is what a guest can read and a time is not: netstack's places end at 103, what one poller watches (`MAX_PLACES`, `userland/netstack/src/main.rs`), so 1,000 idle streams cannot exist, the T14's bench opens no peer that holds streams, and QEMU gives no verdict on time. Exit: a guest test counts netstack's reads of its clients' pipes per frame received beside one bulk stream, with 1 idle stream and with 100, and the line closes when the two counts are equal. +- Every batch of received frames and every deadline ends in a pass over every stream, every transmit opportunity walks every stream twice (once for the count each peer address keeps alive, once to pass those whose connect is not answered), and `Node::next_deadline` reads every stream: linear in streams per batch, as netstack's bridge on smoltcp was per frame. For an idle stream a pass is one `recv_with`, two `status` and one read of its client's pipe, which in netstack is a system call: 1,000 system calls a batch with 1,000 idle streams. A pass's `recv_with` also re-files its connection's deadline and offers it to the transmit round, so one batch makes every idle stream eligible and the next opportunity serves each to learn it has nothing. [tcp] cannot say which connections a segment, a timer, an ICMP error or a failed next hop moved: it reports `drain_eligible` and `drain_gone`, which are the round's. The change: [tcp] marks a held connection in `for_conn`, `tick`, `icmp` and `end` and hands the marks out once (`drain_moved`); the node maps a connection to its stream, keeps its deadlines ordered, and passes only the streams [tcp] moved, the one a pipe's wake names, and those due. A pipe's end-gone answer (`Node::pipe_gone`) runs a pass of its own as well, beside the one `Sockets::bridge` runs for a wake's readiness answers, so k clients leaving in one wake cost k + 1 passes. Not measured at the move. The count is what a guest can read and a time is not: netstack's places end at 103, what one poller watches (`MAX_PLACES`, `userland/netstack/src/main.rs`), so 1,000 idle streams cannot exist, the T14's bench opens no peer that holds streams, and QEMU gives no verdict on time. Exit: a guest test counts netstack's reads of its clients' pipes per batch received beside one bulk stream, with 1 idle stream and with 100, and the line closes when the two counts are equal. - A client that is gone with nothing left in its pipe is finished by [tcp]'s rules for a user who let go: a reset once it has been idle 60 s, where netd on smoltcp reset it 100 s after the client left whatever the peer said. A peer that acknowledges a byte a minute holds such a connection for as long as it has bytes to acknowledge. Exit: the T14 or a guest shows such a peer holding one, and [tcp]'s rule takes a bound from the client's leaving; or the owner rules the idle bound is the one. - `Tcp::ready`, `Tcp::orphans` and `Tcp::options` are additions to [tcp] the specifications do not hold, made for the node's wakes, its places and the copy a stream keeps of its connection's options; their tests (`toyos-net-shard/tcp/tests/held.rs`) name no scenario id. `Shard::listen` refuses an address [ip] does not hold usable, by the rule [udp]'s bind refuses by (`udp.bind-address-not-local`), and counts nothing for it; the node's test of it (`a_listener_is_at_the_address_it_named_and_only_one_the_machine_holds`) names no scenario either. A listener whose address the machine loses stands as a datagram socket bound to it does, with its port and its place and its owner told nothing, and answers again once the address is back (`a_listener_stands_while_its_address_is_lost_and_answers_when_it_is_back`, `userland/netstack/node/tests/lease.rs`); no host was read for it. Exit: the specifications hold the three calls and the refusal, and the tests name their scenarios. - The word each of [udp]'s refusals is answered in on the pipe ABI is `Refused` in `userland/netstack/node/src/datagram.rs`, tested by `each_refusal_is_answered_in_the_pipes_word_for_it` against no reader's scenario; `udp.no-ephemeral-port` has a word and no test that reaches it. Exit: the scenario owed below exists, and the test names its id. @@ -73,6 +73,7 @@ What stage 4 departs from its specifications: - The readers' stage 4 design has a flow waiting for its next hop keep its place in the round. [shard]'s round lets it go, deficit and all, and [tcp] offers it again when woken, at the round's tail, so a waiting flow costs the round nothing. Exit: a `toyos-net-shard/tests/drr.rs` test finds a woken flow served in the place it held when it began to wait. - The readers' stage 4 design has a flow leave the round once it has nothing eligible, its deficit reset. [udp] says so as it hands out a sender's last datagram; [tcp] learns it only when it serves the connection, since only its `next_segment` decides what is due, so a connection whose turn ended on its last segment stays in the round with its deficit until its next turn, and what it queues before then is sent against that deficit, debt included, as RFC 8290 §4.2 keeps an emptied queue until it is next selected. Exit: [tcp] reports at hand-off that nothing is left, and a `toyos-net-shard/tests/drr.rs` test finds such a connection out of the round, its deficit reset. - `Tcp::recv_with`, the read of a reader that may take less than it is shown, is not in the specifications: the node's bridge to a client's pipe needs it, since a byte `recv` copied out has opened the window and cannot be put back. Its tests (`toyos-net-shard/tcp/tests/take.rs`) name no scenario. Exit: the specification names the call and its scenarios, and the tests take their ids. +- RFC 9293 §3.10.7.4 says a reset in a synchronized state SHOULD flush every queue. [tcp] keeps the in-order text received before it, which `recv` and `recv_with` give ahead of `Err(Failed(Reset))` (`Ended`, `toyos-net-shard/tcp/src/stack.rs`), as Linux and the BSDs read buffered data ahead of `ECONNRESET`: the orchestrator ruled to follow them, since ToyOS's clients are written against those stacks, and since text a pass has already written to a client's pipe cannot be flushed, so only keeping it makes what a client reads independent of how the card batched frames. `s_rx_023_a_reset_comes_after_the_text_before_it` holds it. Exit: the specification names the departure, or the owner rules for the flush. - The readers' TCP specification disagrees with itself, and [tcp] follows its user timeout: one replaces R2 and runs only while sent data is unacknowledged or a zero window holds data back, so with one set a connection whose data never left, for want of credit or of a next hop, has no give-up, where its rule that give-up clocks run on wall time has a device that never offers credit end its connections. Exit: RFC 9293 §3.10.8 ends a connection in any state once its user timeout expires: a test that sets one and withholds credit, as `s_pl_009_give_up_without_credit` does without one, finds the connection ended. Open for the owner. On 2026-10-03 he chose "Fair scheduler now": "Build the byte-fair scheduler as the network spec describes, about 280 lines, and decide later on the laptop whether new connections should jump the queue." It is built as one round, in the order flows became eligible, and nobody jumps it. The track reads a queue that new connections jump as RFC 8290's new-flows list, and offers him this evidence: a sparse flow such as a DNS query waits one turn of every flow ahead of it in the round, up to a quantum of 1,514 bytes each, where [udp]'s datagrams and [tcp]'s segments alternated frame by frame before, so a query waited at most one TCP frame: in DRR-02 a 100-byte segment that becomes eligible leaves behind a full frame of each of the three bulk flows ahead of it (`s_shard_drr_002_a_bulk_flow_starves_no_other_beyond_its_quantum`); a T14 measurement at stage 5 of a sparse flow's wait behind bulk flows would add to it. Exit: the owner rules on whether new connections jump the queue. diff --git a/toyos-net-shard/tcp/src/stack.rs b/toyos-net-shard/tcp/src/stack.rs index acb35480fef..6fdd2cf6a83 100644 --- a/toyos-net-shard/tcp/src/stack.rs +++ b/toyos-net-shard/tcp/src/stack.rs @@ -100,7 +100,8 @@ enum User { struct Ended { failure: Option, - /// After both FINs: what the user has still to read. + /// After both FINs or the peer's reset: what the user has still to read, ahead of the failure + /// (Linux and the BSDs, not RFC 9293 §3.10.7.4's SHOULD-flush). rx: Option>, } @@ -474,7 +475,7 @@ impl Tcp { } /// A connection's end: the user who holds it keeps the socket to learn why (and to read what - /// both FINs left); anyone else's is freed. + /// both FINs or a reset left); anyone else's is freed. fn end(&mut self, index: u32, failure: Option, rx: Option>) { let Some(conn) = value(&mut self.conns, index) else { return }; match conn.user { @@ -887,7 +888,7 @@ impl Tcp { self.settle(index, now); } Verdict::Closed => self.end(index, None, Some(Box::new(sync.rx))), - Verdict::Reset => self.end(index, Some(Failure::Reset), None), + Verdict::Reset => self.end(index, Some(Failure::Reset), Some(Box::new(sync.rx))), Verdict::TimeWait => { let tw = TimeWait::from_sync(&sync, now); let owed = sync.rx.ack_now || sync.rx.dup_owed > 0 || sync.rx.delayed.is_some(); @@ -1010,10 +1011,9 @@ impl Tcp { let result = match &mut conn.state { Tcb::SynSent(_) | Tcb::SynRcvd(_) => Err(Error::WouldBlock), Tcb::Sync(sync) => sync.recv(out, now), - Tcb::Ended(Ended { failure: Some(failure), .. }) => Err(Error::Failed(*failure)), - Tcb::Ended(Ended { failure: None, rx }) => match rx.as_mut().map(|rx| rx.read(out)) { + Tcb::Ended(Ended { failure, rx }) => match rx.as_mut().map(|rx| rx.read(out)) { Some(n) if n > 0 => Ok(Received::Data(n)), - _ => Ok(Received::End), + _ => failure.map_or(Ok(Received::End), |f| Err(Error::Failed(f))), }, }; self.settle(id.index, now); @@ -1029,10 +1029,9 @@ impl Tcp { let result = match &mut conn.state { Tcb::SynSent(_) | Tcb::SynRcvd(_) => Err(Error::WouldBlock), Tcb::Sync(sync) => sync.recv_with(take, now), - Tcb::Ended(Ended { failure: Some(failure), .. }) => Err(Error::Failed(*failure)), - Tcb::Ended(Ended { failure: None, rx }) => match rx.as_mut() { + Tcb::Ended(Ended { failure, rx }) => match rx.as_mut() { Some(rx) if rx.unread() > 0 => Ok(Received::Data(rx.read_with(take))), - _ => Ok(Received::End), + _ => failure.map_or(Ok(Received::End), |f| Err(Error::Failed(f))), }, }; self.settle(id.index, now); diff --git a/toyos-net-shard/tcp/tests/acceptability.rs b/toyos-net-shard/tcp/tests/acceptability.rs index 3bdbe23eda7..9ba6f82747d 100644 --- a/toyos-net-shard/tcp/tests/acceptability.rs +++ b/toyos-net-shard/tcp/tests/acceptability.rs @@ -75,7 +75,8 @@ fn s_ac_004_zero_window_pure_ack() { fn s_ac_005_zero_window_rst() { let mut h = zero_window(); nothing(&h.input(10, seg(70_536).rst())); - reset(&mut h); + let status = h.status(); + assert_eq!((status.state, status.failure), (State::Closed, Some(Failure::Reset))); } #[test] diff --git a/toyos-net-shard/tcp/tests/receive.rs b/toyos-net-shard/tcp/tests/receive.rs index b291672a377..bb63a42e364 100644 --- a/toyos-net-shard/tcp/tests/receive.rs +++ b/toyos-net-shard/tcp/tests/receive.rs @@ -230,10 +230,11 @@ fn s_rx_022_no_sack_unnegotiated() { } #[test] -fn s_rx_023_a_reset_is_never_end_of_stream() { +fn s_rx_023_a_reset_comes_after_the_text_before_it() { let mut h = fixture_e(); h.input(1, seg(5001).ack(1001).len(100)); h.input(2, seg(5101).rst()); + assert_eq!(recv(&mut h, 1000), Ok(Received::Data(100))); assert_eq!(recv(&mut h, 1000), Err(Error::Failed(Failure::Reset))); } diff --git a/userland/netstack/node/src/lib.rs b/userland/netstack/node/src/lib.rs index f0506792f6b..420501a525c 100644 --- a/userland/netstack/node/src/lib.rs +++ b/userland/netstack/node/src/lib.rs @@ -150,10 +150,15 @@ impl Node { // ---- the device ---- - /// A frame the device received. - pub fn receive(&mut self, now: Instant, frame: &[u8], mut draw: impl FnMut() -> u32) { - self.stack.receive(now, frame); - self.settle(now, &mut draw); + /// The frames the device received since the last call, one by one: `next` hands its next + /// frame to the sink it is given and answers `false` once it had none. One pass over the + /// streams follows the last frame, so what the batch brought reaches each client in one write + /// and not one a frame. + pub fn receive(&mut self, now: Instant, mut next: impl FnMut(&mut dyn FnMut(&[u8])) -> bool, mut draw: impl FnMut() -> u32) { + while next(&mut |frame| { + self.stack.receive(now, frame); + self.settle(now, &mut draw); + }) {} self.bridge(now); } diff --git a/userland/netstack/node/src/streams.rs b/userland/netstack/node/src/streams.rs index 2266477e275..025a510a42b 100644 --- a/userland/netstack/node/src/streams.rs +++ b/userland/netstack/node/src/streams.rs @@ -37,9 +37,9 @@ //! may take is bounded nowhere. A client that still holds its reading end, or may still write, //! is never timed here: what it cannot send is [tcp]'s to give up on. //! -//! Every call that can move a stream ends in a pass: a frame, a deadline, and each call here. A -//! transmit opportunity can only fail a connect, whose next hop it found to answer nobody, and -//! ends in a pass over the connects. +//! Every call that can move a stream ends in a pass: a batch of frames, a deadline, and each call +//! here. A transmit opportunity can only fail a connect, whose next hop it found to answer +//! nobody, and ends in a pass over the connects. use alloc::boxed::Box; use alloc::collections::{BTreeMap, VecDeque}; diff --git a/userland/netstack/node/tests/common/mod.rs b/userland/netstack/node/tests/common/mod.rs index 16ff2a664df..33c130e1345 100644 --- a/userland/netstack/node/tests/common/mod.rs +++ b/userland/netstack/node/tests/common/mod.rs @@ -291,6 +291,15 @@ pub fn echo_request(destination: Ipv4Addr, id: u16, seq: u16, data: &[u8]) -> Ve ipv4(MAC, MAC_R, R, destination, 1, &icmp) } +/// What [`Node::receive`] hands each frame to. +type Sink<'s> = dyn FnMut(&[u8]) + 's; + +/// `frames`, in order, as the batch [`Node::receive`] takes. +pub fn batch<'a>(frames: &'a [&'a [u8]]) -> impl FnMut(&mut Sink<'_>) -> bool + 'a { + let mut left = frames.iter(); + move |sink| left.next().map(|frame| sink(frame)).is_some() +} + pub struct Wire { pub node: Node, mac: [u8; 6], @@ -341,7 +350,7 @@ impl Wire { } let asked = self.sent[from..].iter().filter(|seen| matches!(seen, Seen::Arp { request: true, sender, target, .. } if *target == R && *sender == A)).count(); for _ in 0..asked { - self.node.receive(self.now, &arp(MAC, false, MAC_R, R, A), draw(&mut self.draws)); + self.node.receive(self.now, batch(&[&arp(MAC, false, MAC_R, R, A)]), draw(&mut self.draws)); } } } @@ -352,7 +361,7 @@ impl Wire { } pub fn deliver(&mut self, frame: &[u8]) { - self.node.receive(self.now, frame, draw(&mut self.draws)); + self.node.receive(self.now, batch(&[frame]), draw(&mut self.draws)); self.pump(); } diff --git a/userland/netstack/node/tests/lan/mod.rs b/userland/netstack/node/tests/lan/mod.rs index bd7e8261809..5d00ae9dd57 100644 --- a/userland/netstack/node/tests/lan/mod.rs +++ b/userland/netstack/node/tests/lan/mod.rs @@ -17,7 +17,7 @@ use etherparse::{ArpOperation, Icmpv4Type, LinkSlice, NetSlice, SlicedPacket, Tr use toyos_net_node::Node; use toyos_net_wire::Instant; -use crate::common::{arp, from_server, message_of, option, sum, terms, xid, Wire, A, ACK, BROADCAST, DISCOVER, ELSEWHERE, MAC, MAC_B, MAC_R, OFFER, R}; +use crate::common::{arp, batch, from_server, message_of, option, sum, terms, xid, Wire, A, ACK, BROADCAST, DISCOVER, ELSEWHERE, MAC, MAC_B, MAC_R, OFFER, R}; pub const B: Ipv4Addr = ELSEWHERE; pub const OFF_LINK: Ipv4Addr = Ipv4Addr::new(198, 51, 100, 7); @@ -161,13 +161,13 @@ impl Lan { .collect(); for target in asked { let mac = if target == R { MAC_R } else { MAC_B }; - self.node.receive(self.now, &arp(MAC, false, mac, target, A), draw(&mut self.draws)); + self.node.receive(self.now, batch(&[&arp(MAC, false, mac, target, A)]), draw(&mut self.draws)); } } } pub fn deliver(&mut self, frame: &[u8]) { - self.node.receive(self.now, frame, draw(&mut self.draws)); + self.node.receive(self.now, batch(&[frame]), draw(&mut self.draws)); self.pump(); } diff --git a/userland/netstack/node/tests/listeners.rs b/userland/netstack/node/tests/listeners.rs index d901574823c..4ecc470ac85 100644 --- a/userland/netstack/node/tests/listeners.rs +++ b/userland/netstack/node/tests/listeners.rs @@ -18,7 +18,7 @@ use std::net::Ipv4Addr; use std::rc::Rc; use std::time::Duration; -use common::{arp, terms, Segment, Wire, A, ELSEWHERE, MAC, MAC_R, R}; +use common::{arp, batch, terms, Segment, Wire, A, ELSEWHERE, MAC, MAC_R, R}; use etherparse::{ArpOperation, LinkSlice, NetSlice, PacketBuilder, SlicedPacket, TcpOptionElement, TransportSlice}; use toyos_net_node::{AcceptRefused, Accepted, ConnectRefused, FromClient, ListenRefused, ListenerId, Node, PipeEnd, Pipes, ReadRefusal, Refused, StreamEvent, StreamId, ToClient, Wake, WriteRefusal}; use toyos_net_tcp::{limits, Counter, Endpoint}; @@ -227,7 +227,7 @@ impl Net { } for frame in &frames { if let Some(answer) = self.hears(frame) { - self.node.receive(self.now, &answer, draw(&mut self.draws)); + self.node.receive(self.now, batch(&[&answer]), draw(&mut self.draws)); } } } @@ -263,7 +263,7 @@ impl Net { } fn deliver(&mut self, frame: &[u8]) { - self.node.receive(self.now, frame, draw(&mut self.draws)); + self.node.receive(self.now, batch(&[frame]), draw(&mut self.draws)); self.pump(); } @@ -808,7 +808,7 @@ fn a_listener_ended_for_its_wake_gives_its_place_to_another_in_the_same_pass() { refusing.borrow_mut().refusal = Some(WriteRefusal::Gone); net.syn(P3); let ack = net.iss(P3).wrapping_add(1); - net.node.receive(net.now, &frame(P3, SSH, ISS + 1, Some(ack), 0, &[]), draw(&mut net.draws)); + net.node.receive(net.now, batch(&[&frame(P3, SSH, ISS + 1, Some(ack), 0, &[])]), draw(&mut net.draws)); assert_eq!(wakes(&owner), 2); let ended: Vec<(ListenerId, WriteRefusal)> = net.node.drain_ended_listeners().collect(); assert_eq!((ended, net.node.listeners(), net.node.held()), (vec![(first, WriteRefusal::Gone)], 1, 1)); diff --git a/userland/netstack/node/tests/streams.rs b/userland/netstack/node/tests/streams.rs index 699296b573c..7c521334881 100644 --- a/userland/netstack/node/tests/streams.rs +++ b/userland/netstack/node/tests/streams.rs @@ -16,7 +16,7 @@ use std::net::Ipv4Addr; use std::rc::Rc; use std::time::Duration; -use common::{arp, terms, Segment, Wire, A, MAC, MAC_B, MAC_R, R}; +use common::{arp, batch, terms, Segment, Wire, A, MAC, MAC_B, MAC_R, R}; use etherparse::{ArpOperation, LinkSlice, NetSlice, PacketBuilder, SlicedPacket, TcpOptionElement, TransportSlice}; use toyos_net_node::{ConnectRefused, FromClient, Node, PipeEnd, Pipes, ReadRefusal, StreamEvent, StreamId, ToClient, Watch, WriteRefusal}; use toyos_net_shard::ConnectError; @@ -62,6 +62,9 @@ struct Ends { from_dropped: bool, /// The ends the node let go of, in the order it did. let_go: Vec, + /// The node's calls of each end: each is a syscall in netstack. + writes: usize, + reads: usize, } type Client = Rc>; @@ -72,6 +75,7 @@ struct ReadEnd(Client); impl ToClient for WriteEnd { fn write(&mut self, bytes: &[u8]) -> Result { let mut ends = self.0.borrow_mut(); + ends.writes += 1; if ends.write_broken { return Err(WriteRefusal::Broken); } @@ -98,6 +102,7 @@ impl Drop for WriteEnd { impl FromClient for ReadEnd { fn read(&mut self, out: &mut [u8]) -> Result { let mut ends = self.0.borrow_mut(); + ends.reads += 1; if ends.read_broken { return Err(ReadRefusal::Broken); } @@ -417,14 +422,14 @@ impl Net { self.node.transmit(self.now, usize::MAX, |frame| frames.push(frame.to_vec()), draw(&mut self.draws)); for frame in &frames { for answer in self.hears(frame) { - self.node.receive(self.now, &answer, draw(&mut self.draws)); + self.node.receive(self.now, batch(&[&answer]), draw(&mut self.draws)); } } frames.len() } fn deliver(&mut self, frame: &[u8]) { - self.node.receive(self.now, frame, draw(&mut self.draws)); + self.node.receive(self.now, batch(&[frame]), draw(&mut self.draws)); self.pump(); } @@ -633,14 +638,18 @@ fn a_connect_closed_before_its_answer_is_answered_closed() { assert_eq!((net.far.segments.len(), net.far.resets.len()), (sent, 0)); } -// A frame and a deadline each end in a pass of their own: an opportunity's pass is over the -// connects only, so neither test offers one. +// A batch of frames and a deadline each end in a pass of their own: an opportunity's pass is over +// the connects only, so neither test offers one. The batch's pass is one, after its last frame: +// what the batch brought is one write to the client's pipe, and its send pipe is asked once. #[test] -fn a_frame_moves_an_established_stream_with_no_opportunity_after_it() { +fn a_batch_of_frames_moves_an_established_stream_once_with_no_opportunity_after_it() { let (mut net, _, client) = established(); - let frame = net.far.text(b"at once"); - net.node.receive(net.now, &frame, draw(&mut net.draws)); - assert_eq!(client.borrow().inbox, b"at once"); + let frames = [net.far.text(b"all "), net.far.text(b"at "), net.far.text(b"once")]; + let (writes, reads) = (client.borrow().writes, client.borrow().reads); + net.node.receive(net.now, batch(&[&frames[0], &frames[1], &frames[2]]), draw(&mut net.draws)); + let ends = client.borrow(); + assert_eq!(ends.inbox, b"all at once"); + assert_eq!((ends.writes - writes, ends.reads - reads), (1, 1), "the pipes' calls in the batch's pass"); } #[test] @@ -1009,6 +1018,35 @@ fn a_reset_ends_both_pipes_and_the_stream() { assert!(net.events().is_empty() && net.far.resets.is_empty()); } +// Text and the reset after it in one batch: the client reads the text before the failure, as it +// would had they come in passes of their own (Linux and the BSDs read what arrived before the +// reset ahead of ECONNRESET). +#[test] +fn a_reset_in_the_batch_of_the_text_before_it_ends_the_pipes_after_the_text() { + let (mut net, _, client) = established(); + let text = net.far.text(b"last"); + let rst = net.far.rst(); + net.node.receive(net.now, batch(&[&text, &rst]), draw(&mut net.draws)); + assert_eq!((client.borrow().inbox.clone(), let_go(&client)), (b"last".to_vec(), vec![PipeEnd::FromClient, PipeEnd::ToClient])); +} + +// The reset's text outlives the pass that saw the reset while the client's pipe has no room for +// it, and the failure is read after it. +#[test] +fn text_the_pipe_had_no_room_for_outlives_the_reset() { + let (mut net, _, client) = established(); + client.borrow_mut().room = 0; + let text = net.far.text(b"held back"); + let rst = net.far.rst(); + net.node.receive(net.now, batch(&[&text, &rst]), draw(&mut net.draws)); + assert_eq!((client.borrow().inbox.len(), let_go(&client), net.node.streams()), (0, vec![PipeEnd::FromClient], 1)); + + client.borrow_mut().room = 65_536; + net.bridge(); + assert_eq!(client.borrow().inbox, b"held back"); + assert_eq!((let_go(&client), net.node.streams()), (vec![PipeEnd::FromClient, PipeEnd::ToClient], 0)); +} + // ---- a client that leaves ---- #[test] diff --git a/userland/netstack/src/main.rs b/userland/netstack/src/main.rs index 014e6615b87..bc588329fbf 100644 --- a/userland/netstack/src/main.rs +++ b/userland/netstack/src/main.rs @@ -9,7 +9,7 @@ //! random source, the kernel's pipes (`pipes`), the clients' connections //! (`client`) and their requests (`serve`). //! -//! **One pass**: the card's link and its received frames go to the node, then +//! **One pass**: the card's link and its received frames, as one batch, go to the node, then //! every deadline that is due, then the node is offered the card's transmit //! room until it has no frame left or the card no room; what the node has to //! say is written to the log and to the clients that waited for it; and the @@ -380,7 +380,7 @@ fn main() { node.link(clock(), link_up, draw); } } - while card.rx(|frame| node.receive(clock(), frame, draw)) {} + node.receive(clock(), |sink| card.rx(sink), draw); let now = clock(); if node.next_deadline().is_some_and(|at| at <= now) { node.fire(now, draw);