From 83ef5a9ae9ee9563c641ae3ff9ea69e85b46baae Mon Sep 17 00:00:00 2001 From: japabu Date: Fri, 9 Oct 2026 21:48:03 +0200 Subject: [PATCH 1/3] The T14 downloads over HTTPS through ToyOS's own stack inside its list's bound, and netstack's 64 KiB receive window is filed as the ceiling its rate sits on The owner asked whether ToyOS reaches gigabit and ruled the measurement an internet download. `https_download` fetches one public file with the client `https_get` builds, now one module both read (`https_client.rs`): an unchanged ureq on rustls on ring, the project's User-Agent and nothing else identifying. The file is Rust 1.81.0's x86_64 Linux archive on static.rust-lang.org, 170439044 bytes; the `internet_download` metal row holds a whole body to that length and the SHA-256 the project publishes beside it, so the job holds no pin and prints `sha256=` as `https_get` does. The transfer is bounded by time, not by size. The runner hands each job of a list the bound it gave the list (`toyos_tco::LIST_BOUND_ENV`), and the job's window is that bound less 5 s for its line to reach the stick. A body read to its end says `whole`; one the window closes on says `cut` with what came, and both end 0. The row's verdict is the transport: the job ended 0, some of the body came, a whole body equals the pin, and the T14's MPERF was read. Its rate is printed and never judged. Before the transfer the job times five TCP handshakes to the file's host, `rtt_ms`: the round trip measured in ToyOS, since netstack exposes no connection's SRTT. The first T14 boot read 35.1 Mb/s at 1.4 % busy, which is 65,535 bytes every 14.9 ms: netstack's per-connection receive buffer (`TCP_BUFFER`) with a window shift of 0, one window per round trip. That is filed as the stack's ceiling under the network track, with an exit a T14 reading can fail. Until a boot reads `rtt_ms` beside the rate, the window is the ceiling by arithmetic and not by measurement, and the rate is said to be neither the line's nor the path's. netstack's lease line is declared once in toyos-tco, which netstack says and the harness and the job read. The guest test of this job is cut: the HTTPS GET with the project's User-Agent against the harness's server is `https_fetch`'s, its lease wait cannot be reached in QEMU, and its counters read nothing there. The records that said no T14 row reads the wired card, and the mDNS claim on it unread, now say what the row read. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C --- ...from-boot-whatever-its-runner-was-given.md | 2 +- ...ion-at-one-64-kib-window-per-round-trip.md | 39 ++++ ...annot-reach-the-t14-while-it-runs-toyos.md | 9 +- ...cs-room-and-wake-are-unread-on-hardware.md | 5 +- issues/the-lan-is-not-yet-production-grade.md | 3 +- issues/toyos-has-its-own-network-stack.md | 7 +- src/build.rs | 1 + tests/downloadcase/system.toml | 43 ++++ .../src/bin/https_download.rs | 188 ++++++++++++++++++ tests/toyos-rust-tests/src/bin/https_get.rs | 24 +-- tests/toyos-rust-tests/src/https_client.rs | 27 +++ tests/toyos.rs | 51 ++++- toyos-tco/src/lib.rs | 11 + userland/netstack/src/main.rs | 8 +- userland/test-runner/src/main.rs | 12 +- 15 files changed, 391 insertions(+), 39 deletions(-) create mode 100644 issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md create mode 100644 tests/downloadcase/system.toml create mode 100644 tests/toyos-rust-tests/src/bin/https_download.rs create mode 100644 tests/toyos-rust-tests/src/https_client.rs diff --git a/issues/acpi-hold-gives-up-at-a-time-counted-from-boot-whatever-its-runner-was-given.md b/issues/acpi-hold-gives-up-at-a-time-counted-from-boot-whatever-its-runner-was-given.md index c1283ae44fe..f68c188321b 100644 --- a/issues/acpi-hold-gives-up-at-a-time-counted-from-boot-whatever-its-runner-was-given.md +++ b/issues/acpi-hold-gives-up-at-a-time-counted-from-boot-whatever-its-runner-was-given.md @@ -36,5 +36,5 @@ The harness: `tests/toyos-rust-tests/src/bin/acpi_hold.rs`, and ## Exit The hold is bounded from its own start, or by the bound its runner was given, -and a job added before it cannot red `acpi_server_events` over a line the log +which the runner hands each job of a list in `toyos_tco::LIST_BOUND_ENV`, and a job added before it cannot red `acpi_server_events` over a line the log holds. diff --git a/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md b/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md new file mode 100644 index 00000000000..2d583243999 --- /dev/null +++ b/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md @@ -0,0 +1,39 @@ +--- +status: open +kind: defect +opened: 2026-10-09 +--- + +# netstack's receive buffer caps every connection at one 64 KiB window per round trip + +netstack gives each connection a receive buffer of 65,535 bytes (`TCP_BUFFER`, +`userland/netstack/src/main.rs`), and the window it offers a sender is never +more than that buffer. `toyos-net-tcp` negotiates RFC 7323 window scaling, but +takes the smallest shift that fits the buffer in 16 bits (`Tcp::new`, +`toyos-net-shard/tcp/src/stack.rs`), which for this buffer is 0. A sender +then has at most 65,535 bytes in flight, and a download reaches at most +65,535 bytes per round trip, whatever the link: about 35 Mb/s at 15 ms, +5 Mb/s at 100 ms. + +A larger buffer is paid in places: `PLACE_BYTES` counts a listener's +`LISTEN_READY` receive buffers and a stream's two, out of the eighth of memory +netstack's places may hold. + +## Measured + +The T14, `internet_download` at `b323e70ee`, one boot, the link up at +1000 Mb/s: `whole bytes=170439044 secs=38.826 mbps=35.1 busy=0.014`. +35.1 Mb/s is 65,535 bytes every 14.9 ms, and the CPUs were 1.4 % busy across +the transfer. That boot did not read the connection's round trip, so the +window is the ceiling by arithmetic and not yet by measurement; the job says +`rtt_ms`, each of its handshakes' times, from the next boot on. netstack exposes no connection's SRTT: its `inspect` +snapshot counts sockets and nothing of one. + +## Owner + +`issues/toyos-has-its-own-network-stack.md`. + +## Exit + +A T14 `internet_download` boot whose `mbps` exceeds 524.3 over the smallest +of its `rtt_ms` (65,535 bytes a round trip, in Mb/s). diff --git a/issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md b/issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md index 936917fbaa0..1296dc85d96 100644 --- a/issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md +++ b/issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md @@ -4,7 +4,7 @@ kind: tooling opened: 2026-10-08 --- -# The host cannot reach the T14 while it runs ToyOS, and no T14 row reads its wired card +# The host cannot reach the T14 while it runs ToyOS The development host reaches the T14 only while it runs Ubuntu. Under ToyOS no connection the host opens to it arrives, so every row whose boot the host had @@ -65,11 +65,12 @@ the tree has is the flat sleep ## What stands in the meantime -No T14 row reads netstack on the I219, and none reads a message the card +One T14 row reads netstack on the I219, `internet_download`, which judges one +HTTPS download and no count of the card's; none reads a message the card raised. `claim_reuses_its_remapping_entry` claims the function and gives it back twice and takes no traffic. So the calls #763 moved onto a claim's -binding — the I219's interrupt records, its polls and its DMA grants, driven -at rate — have not been read on real hardware since they landed; under QEMU +binding — the I219's interrupt records, its polls and its DMA grants — have +carried that download on real hardware and been read by nothing else; under QEMU `iommu_virtio_platform` and the suite's network tests drive the same `pcidev` functions through virtio-net's claim. diff --git a/issues/the-intel-nics-room-and-wake-are-unread-on-hardware.md b/issues/the-intel-nics-room-and-wake-are-unread-on-hardware.md index abac3c28a82..18e9c0f7d88 100644 --- a/issues/the-intel-nics-room-and-wake-are-unread-on-hardware.md +++ b/issues/the-intel-nics-room-and-wake-are-unread-on-hardware.md @@ -10,8 +10,9 @@ netstack offers its card a frame only when the driver counts room for it, and where there is none it sleeps on the claim until `toyos_i219::I219::wake_on_room`'s cause arrives. On the I219 and the 82574 every reading of that path is the driver's own model (`toyos-i219/src/stub.rs`), which is Intel's documents -written down: no image gives netstack `pci:8086:15fc`, and the harness has no -Intel card, so neither the T14 nor QEMU's `e1000e` has run it. +written down: the one image that gives netstack `pci:8086:15fc`, +`tests/downloadcase`, reads none of the counters below, and the harness has no +Intel card, so neither the T14 nor QEMU's `e1000e` has been read running it. What the documents do not say of the T14's part, and the model therefore takes on trust: that the PCH's MAC raises a message for `TXDW` once it is unmasked. diff --git a/issues/the-lan-is-not-yet-production-grade.md b/issues/the-lan-is-not-yet-production-grade.md index 744210c307f..22e9d7a2d6d 100644 --- a/issues/the-lan-is-not-yet-production-grade.md +++ b/issues/the-lan-is-not-yet-production-grade.md @@ -86,7 +86,8 @@ of real traffic with no error, leak or stall. ## Standing The bench tests that prove each exit run in the T14 suite on every change to -the path, not once. Today none does: no T14 row reads the wired card +the path, not once. Today none does: the one T14 row that reads the wired +card, `internet_download`, judges none of these exits (`issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md`). The bench itself stays honest: the boot stick's first command still sometimes goes unanswered on the USB2 half of its receptacle (runs 84, 103, 104, 105), and everything above rests on the record it carries. diff --git a/issues/toyos-has-its-own-network-stack.md b/issues/toyos-has-its-own-network-stack.md index 33327657315..ec0820e6512 100644 --- a/issues/toyos-has-its-own-network-stack.md +++ b/issues/toyos-has-its-own-network-stack.md @@ -15,13 +15,14 @@ netstack ran smoltcp until stage 5. Its replacement is ToyOS's own stack, built - Stage 4: `toyos-net-shard` and `toyos-net-testnet`. - Stage 5: netd on one shard, pipe ABI unchanged; smoltcp leaves netd, `Cargo.toml` and `Cargo.lock` in the same PR. - Then multi-core, netring (blocked on the owner's ABI ruling), TCP and IP hardening, IPv6, offloads, soak. -- This track owns the T14's outbound rows, the machine reaching its router and the internet on the I219, judged from the stick: they arrive on this stack and not on smoltcp (owner: "no smoltcp."), and until they do no T14 row reads the wired card (`issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md`). +- This track owns the T14's outbound rows, the machine reaching its router and the internet on the I219, judged from the stick: they arrive on this stack and not on smoltcp (owner: "no smoltcp."). The first is `internet_download`, one HTTPS download from the internet, the one T14 row that reads the wired card (`issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md`). Stage 5 lands in slices, the orchestrator's cut under the owner's words "i want smoltcp out as fast as possible": netd's decisions go into `toyos-net-node` (`userland/netstack/node`), pure and host-tested and shipped in nothing, and then one change moves netd onto it and deletes smoltcp. In the tree: the node, its DHCP lease, its clients' datagram sockets and their broadcast permission, its mDNS name, its resolver, streams, listeners and the one bound on its clients' streams, listeners and datagram sockets (`userland/netstack/node/src/places.rs`); a lookup's sockets stand outside that bound, under `toyos_dns::MAX_LOOKUPS` and its rounds; and datagram senders that wait for their next hop in their own queue, with [ip] taking none of them. The move is in the tree too: `userland/netstack` is the card, the clock, the kernel's random source and pipes and the clients' requests around the node (`userland/netstack/src/main.rs`), and smoltcp is in no manifest and no lockfile. What the node does not yet meet: -- Of what a third party wrote, it has met QEMU's user network in a guest, on virtio and on QEMU's e1000e: its DHCP server, its ARP and the host kernel's TCP behind it (`netstack_streams`, `netstack_socket_churn`, `libc_sockets`). It has met no router and no switch. Exit: the T14's outbound rows are green on it. +- Of what a third party wrote, it has met QEMU's user network in a guest, on virtio and on QEMU's e1000e: its DHCP server, its ARP and the host kernel's TCP behind it (`netstack_streams`, `netstack_socket_churn`, `libc_sockets`). On the T14 it has met one router, its DHCP server and a CDN behind it: `internet_download` green at `b323e70ee`, one boot. Exit: the T14's outbound rows are green on it. +- A connection's receive window is at most 65,535 bytes, so a download takes at most that per round trip: `issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md`. - It counts a DHCP message [udp] refused as `node.dhcp-unsent`, whatever the rule; the `dhcp.renew-unroutable` scenario owed above is not written. Exit: that scenario names the counter, or the node counts the renewal apart. - With the link down the DHCP client keeps its timers: every 4 to 64 s the node has a deadline, builds a DISCOVER that [udp] refuses, and counts it in `dhcp.tx.discover` and `node.dhcp-unsent`, from `Node::new` on. Exit: `toyos-dhcp`'s client is told the link went down and waits for it, and the node's first DISCOVER is the one that leaves. - The node's link-down handling (`Node::link`, `userland/netstack/node/src/lib.rs`) and its INIT-REBOOT, a held lease asked for again by REQUEST when the link returns (`Client::link_up`, `toyos-dhcp/src/lib.rs`), have never run on a real card or against a real DHCP server: host tests alone hold them, and no T14 boot has had its link go. Exit: one attended T14 boot with the cable pulled and returned, whose log shows `netstack: I219: link down`, then on the link's return the held lease verified by a REQUEST and its ACK, or taken again, with neither DHCP line that says this machine has no address, then the name's claim line, `netstack: mDNS: no host answered for toyos-t14.local; this machine answers as it`, again. @@ -38,7 +39,7 @@ What the node does not yet meet: - The word each of [udp]'s refusals is answered in on the pipe ABI is `Refused` in `userland/netstack/node/src/datagram.rs`, tested by `each_refusal_is_answered_in_the_pipes_word_for_it` against no reader's scenario; `udp.no-ephemeral-port` has a word and no test that reaches it. Exit: the scenario owed below exists, and the test names its id. - A client's datagram to a broadcast address needs its socket's permission, where smoltcp sends it for every socket. The owner's ruling: "Of course carry the permission why would you even ask. The premise is and always has been to do things properly". The pipe ABI carries the permission as far as netd: `MsgType::UdpSetOption` with `OPT_BROADCAST` (`toyos/src/net.rs`), sent by std's `set_broadcast` (`sdk/std/sys/net/connection.rs`) and by libc's `setsockopt` of `SO_BROADCAST` (`userland/libc/src/socket.rs`), which keeps it for a socket not yet bound and sends it at `bind`; the word for a send refused for want of it is `ERR_PERMISSION_DENIED`, `ErrorKind::PermissionDenied` in std and `EACCES` in libc. The node enforces it, and netstack hands it the request (`Sockets::udp_set_option`, `userland/netstack/src/serve.rs`): `Node::udp_set_broadcast` hands the permission to `Udp::set_broadcast`, and a send without it is refused `udp.broadcast-not-permitted` and answered `Refused::PermissionDenied` (`userland/netstack/node/src/datagram.rs`), which `a_broadcast_needs_its_permission` (US-21, `userland/netstack/node/tests/datagram.rs`) holds, with both broadcast addresses read off the wire in link-broadcast frames once the socket holds it and refused again once it is taken back, and `each_refusal_is_answered_in_the_pipes_word_for_it` holds the word. The permission of the call goes with the datagram to the route it leaves by: [udp] queues it with what its socket held, a closed socket's included, and `Ip::send_udp` refuses a link broadcast to one without it, `ip.broadcast-not-permitted`, counted and logged, so a prefix a DHCP server renews under a held address between the call and the frame makes no broadcast of a datagram accepted for a host (`s_udp_us_021_shard_a_datagram_accepted_for_a_host_is_no_broadcast_after_a_renewal_narrows_the_prefix` and its two neighbours in `toyos-net-shard/tests/udp.rs`, and from a server's ACK `a_datagram_accepted_for_a_host_is_no_broadcast_after_the_server_renews_a_narrower_prefix` in the node's tests). The search is run. `grep -rnI -E 'SO_BROADCAST|INADDR_BROADCAST|255\.255\.255\.255|sendto|setsockopt|sys/socket\.h|socket\(' userland/doom/doomgeneric`, the one third-party C program an image builds, at the commit `userland/doom/build.rs` pins, exits 1 with no line; the same without `sys/socket\.h` over `tests/testcases` exits 1 with no line; `SO_BROADCAST|INADDR_BROADCAST|255\.255\.255\.255` over the trees of `src/libcxx.rs`'s `SOURCES` finds LLVM libc's own macro and its documentation and no caller; and in ToyOS's tree outside the stack's crates the option is named only by the two setters. The socket2 fork's setter reaches nobody: `issues/the-socket2-forks-so-broadcast-setter-does-nothing.md`. Exit: a guest test, `udp_broadcast_needs_its_permission`, reads `broadcast()` false, then true after `set_broadcast(true)`, on the socket and on a `try_clone` of it, which `broadcast()` answering a constant or a duplicate holding a value of its own turns red, and sends to the limited broadcast address before and after the set and is refused `PermissionDenied` only before, which std's setter reverted to `Ok(())` turns red; and a guest C case reads libc's setter, `socket`, `setsockopt(SO_BROADCAST)`, `bind`, `sendto` to the limited broadcast address, refused `EACCES` without the option and sent with it, and with the option set before `bind`, which `bind` no longer handing the kept option over turns red. The case runs the sequence without `bind` too, where the first `sendto` binds the socket and hands the kept option over; `tests/netcase/sendto_unbound.c` holds that send for a socket permitted to make it. - The machine's name is told to the node twice, as `toyos_dhcp::HostName` in `Node::new` and as `toyos_mdns::Host` in `Node::answer_as`: the first owns its bytes and the second borrows them. netstack's is a constant, so neither takes a leak. Exit: one type carries the label to both. -- The machine's name is probed for and defended (RFC 6762 §8, §9) by `toyos-mdns` (`userland/netstack/mdns`), on the node, which netstack runs. What that does not yet meet. Every machine is named `toyos-t14` (`HOSTNAME`, `userland/netstack/src/main.rs`) and a responder that loses its name holds none, so of two on one network the second answers to no name: asked "When two machines on one network have the same ToyOS host name, what should the second one do?", the owner answered "Hold no name, say so (Recommended)". Asked on 2026-10-09 "A ToyOS machine that loses its .local name to a conflict stays nameless until its network link next returns, even after the other machine has left. Should it try for its name again by itself?", he answered "Retry on a slow interval (Recommended)". Built: a lost name is probed for again a minute after each loss (`RETRY_MS`, `userland/netstack/mdns/src/lib.rs`) and at once on a link after none, and is claimed by the first probing no host answers; RFC 6762 permits probing again at §8.1's rate ("wait five seconds after any failed probe attempt before trying again") and does not require it. Two packets from any host on the link still take a held name, a response to take it back to probing and a response under the probe, and one more a minute keeps it: no path back exists that a peer cannot hold shut, since a host that answers every probe is what a holder of the name is. The caller does not say how a message was addressed, so two rules that turn on it are not applied: a response is read however it came, where §6 reads a unicast one only within two seconds of a question that asked for one; and a message from a source outside the subnet is ignored even when it was sent to the group, where §11 deems that on the link "regardless of source IP address", so a host of the same name on another subnet of the link is never a conflict. Exit: each machine's name is its own, and a test with two guests on one network finds each by its name; and the caller says how a message was addressed, with a test in which a unicast response nobody asked for takes no name and one in which a response sent to the group from another subnet does. Not measured, each with its exit: the shipped responder's claim on the T14's wired card, which no `system.toml` in the tree gives netstack (when the outbound rows land, their boot shows the lease line, then the claim line, and no loss line); and the link's return on metal (the orchestrator's attended session with the owner, never automated). A link that goes and returns inside one pass of the Intel driver reaches the name as no change: `issues/a-link-that-goes-and-returns-inside-one-pass-of-the-intel-driver-is-reported-as-no-change.md`. +- The machine's name is probed for and defended (RFC 6762 §8, §9) by `toyos-mdns` (`userland/netstack/mdns`), on the node, which netstack runs. What that does not yet meet. Every machine is named `toyos-t14` (`HOSTNAME`, `userland/netstack/src/main.rs`) and a responder that loses its name holds none, so of two on one network the second answers to no name: asked "When two machines on one network have the same ToyOS host name, what should the second one do?", the owner answered "Hold no name, say so (Recommended)". Asked on 2026-10-09 "A ToyOS machine that loses its .local name to a conflict stays nameless until its network link next returns, even after the other machine has left. Should it try for its name again by itself?", he answered "Retry on a slow interval (Recommended)". Built: a lost name is probed for again a minute after each loss (`RETRY_MS`, `userland/netstack/mdns/src/lib.rs`) and at once on a link after none, and is claimed by the first probing no host answers; RFC 6762 permits probing again at §8.1's rate ("wait five seconds after any failed probe attempt before trying again") and does not require it. Two packets from any host on the link still take a held name, a response to take it back to probing and a response under the probe, and one more a minute keeps it: no path back exists that a peer cannot hold shut, since a host that answers every probe is what a holder of the name is. The caller does not say how a message was addressed, so two rules that turn on it are not applied: a response is read however it came, where §6 reads a unicast one only within two seconds of a question that asked for one; and a message from a source outside the subnet is ignored even when it was sent to the group, where §11 deems that on the link "regardless of source IP address", so a host of the same name on another subnet of the link is never a conflict. Exit: each machine's name is its own, and a test with two guests on one network finds each by its name; and the caller says how a message was addressed, with a test in which a unicast response nobody asked for takes no name and one in which a response sent to the group from another subnet does. The shipped responder's claim on the T14's wired card is read once: the `internet_download` boot at `b323e70ee` shows the lease line, then `netstack: mDNS: no host answered for toyos-t14.local; this machine answers as it`, and no loss line. Not measured, with its exit: the link's return on metal (the orchestrator's attended session with the owner, never automated). A link that goes and returns inside one pass of the Intel driver reaches the name as no change: `issues/a-link-that-goes-and-returns-inside-one-pass-of-the-intel-driver-is-reported-as-no-change.md`. - A socket's datagrams that wait for a next hop wait in the one queue it has, `toyos_net_udp::limits::TX_DATAGRAMS` of them: with all 16 waiting for one next hop, the socket's sends to every other destination are refused `udp.tx-queue-full`, `Refused::ResourceExhausted` to a client, until that hop answers or [ip] gives it up, 3 s after its first request left (`BROADCAST_SOLICIT` requests a `RETRANS` apart). It is no regression by this track's earlier record of what ships, that smoltcp kept each datagram in its own socket until the neighbour answered. What a closed socket had accepted waits in the closed sender's `CLOSED_DATAGRAMS` the same way, so 16 datagrams of closed sockets that wait for one silent host have the next closes' datagrams discarded, `udp.tx-discarded-on-close`, for those 3 s: the line on US-53 among stage 3's departures. `s_udp_us_025_what_waits_for_a_next_hop_is_bounded_by_its_sockets_queue` holds the first as it stands. The move's guest runs showed no program refused for it: none of their jobs sends sixteen datagrams past a hop that does not answer. Owner: whoever reads the T14's first outbound run on this stack. Exit: a run on the T14 or in a guest shows a program refused for it, and a socket's bound is counted per next hop; or the owner rules the one queue is the bound. - A link that goes down drops no waiting datagram by itself: every waiting one is handed to [ip] again at the next transmit opportunity, which refuses it for want of a route (`route.no-source-address`), and one for which the link is back before that opportunity waits for its next hop anew, where the readers' link-down rule (NUD-23) drops what waits at once. `s_udp_us_024_shard_a_waiting_datagram_whose_route_goes_is_dropped` holds the first as it stands, and `a_waiting_datagram_whose_link_returns_before_an_opportunity_waits_anew_and_leaves`, beside it in `toyos-net-shard/tests/udp.rs` and under no scenario's id, the second. Not done before the move, whose worker changed nothing below netstack. Owner: whoever next changes `toyos-net-udp`. Exit: that second test's last assertion is inverted and finds the datagram that waited gone, or the specifications have it wait. - `toyos_dns::Lookup::on_datagram` still takes a reply's source address and port, which the node's connected sockets have already matched and the node hands it from its own record of the query: the resolver that read them left with smoltcp, and the node hands both from its own record. Not removed at the move, which left `toyos-dns` as it was. Exit: the two parameters and the comparison inside `on_datagram` go. diff --git a/src/build.rs b/src/build.rs index c96861e52a3..be4629579b1 100644 --- a/src/build.rs +++ b/src/build.rs @@ -2907,6 +2907,7 @@ mod tests { "diag/system.toml", "console/system.toml", "tests/acpicase/system.toml", + "tests/downloadcase/system.toml", "tests/jobcase/system.toml", "tests/latencycase/system.toml", "tests/logstallcase/system.toml", diff --git a/tests/downloadcase/system.toml b/tests/downloadcase/system.toml new file mode 100644 index 00000000000..ead213477b8 --- /dev/null +++ b/tests/downloadcase/system.toml @@ -0,0 +1,43 @@ +# One HTTPS download through netstack, timed: `https_download`'s boot on the +# T14, where the `internet_download` metal row reads its rate off the stick. + +[boot] +start = ["logkeeper", "diskserver", "fileserver", "netstack", "test-runner"] + +# `log` is the port the job waits for netstack's lease on: on the T14 it is the +# boot's first job. +[programs.logkeeper] +service = true +syscap = ["logread"] +serves = ["log"] + +# The T14's onboard I219, by vendor and device. +[programs.netstack] +service = true +serves = ["netstack"] +devices = ["pci:8086:15fc"] + +# `netstack` for the job's lookup and connection, `log` for netstack's word on +# its lease; `counters` and `trace` for every CPU's MPERF and stamp either +# side of the transfer, MPERF being answered only beside `trace`; `dup` +# because the runner hands the job a duplicate. +[programs.test-runner] +receives = ["netstack", "log"] +syscap = ["counters", "trace", "dup"] + +# The block service: the NVMe controller this machine's DATA is on, driven +# from userland through its claim. Started again when it ends; the claim goes +# back with the process and is minted again for the next. +[programs.diskserver] +service = true +restart = true +serves = ["block"] +devices = ["pci:1b36:0010"] + +# The file servers: DATA, the log and the running slot's volume, one process +# each, serving every program the directories of its role. Started again when +# one ends, on the same ports. +[programs.fileserver] +restart = true +roles = ["data", "log", "boot"] +receives = ["block"] diff --git a/tests/toyos-rust-tests/src/bin/https_download.rs b/tests/toyos-rust-tests/src/bin/https_download.rs new file mode 100644 index 00000000000..8427d40ff7e --- /dev/null +++ b/tests/toyos-rust-tests/src/bin/https_download.rs @@ -0,0 +1,188 @@ +//! How fast this machine downloads over HTTPS through ToyOS's own stack: one +//! large public file ([`HOST`], [`PATH`]) with the client `https_get` builds, +//! every byte hashed. The `internet_download` metal row runs it, holds a whole +//! body to the file's pin and reports the rate, never judging it. +//! +//! It runs only in a job list, and its window is the bound the runner gave +//! that list (`toyos_tco::LIST_BOUND_ENV`) less [`MARGIN_MS`]. Inside it, it +//! waits for netstack's lease on the `log` port, since on the T14 it is the +//! boot's first job; times [`PROBES`] TCP handshakes to the file's host; and +//! reads the body until it ends or the window closes. +//! +//! It says one line and ends 0: `https_download: whole bytes= +//! sha256= ` for a body read to its end, or `https_download: cut +//! bytes= ` for what came inside the window, and `https_download: +//! cut before the first byte of the body` for nothing. `` is +//! `secs= mbps= rtt_ms=[] busy= cpus=[]`: the +//! time from the first read of the body to the last and the rate over the +//! bytes after the first; each handshake's time from `connect` called to +//! returned, which is one round trip to the host and netstack's own time for a +//! connect; and the busy fraction each CPU's MPERF ran of its stamp across the +//! transfer (`busy=unread` where no CPU counts MPERF). An I/O or TLS error +//! panics. + +use std::io::Read; +use std::net::{TcpStream, ToSocketAddrs}; +use std::sync::mpsc; +use std::sync::{Mutex, PoisonError}; +use std::time::{Duration, Instant}; + +use toyos::endow::{Endowments, SYSCAP_LABEL}; +use toyos::syscap::SysCap; +use toyos_abi::counters::{Counter, RawRecord, Record}; +use toyos_abi::syscall; +use toyos_logstream::program_line; + +#[path = "../https_client.rs"] +mod https_client; +#[path = "../served_log.rs"] +mod served_log; + +/// A release archive Rust's distribution server never rewrites; the row holds +/// its length and SHA-256. +const HOST: &str = "static.rust-lang.org"; +const PATH: &str = "/dist/2024-09-05/rust-1.81.0-x86_64-unknown-linux-gnu.tar.xz"; +const ROOTS: &str = "/system/etc/ssl/cert.pem"; + +/// Handshakes timed before the transfer, one after another. +const PROBES: usize = 5; + +/// The ceiling on netstack's word reaching this job: its own bound on saying +/// it, and two of logkeeper's rounds at its write budget +/// (`userland/logkeeper/src/policy.rs`, 5 s), since a served line is one the +/// stick already holds. +const LEASE_WAIT: Duration = Duration::from_millis(toyos_tco::LEASE_BOUND_MS + 10_000); + +/// What the window leaves the list's bound for the last line to reach the +/// stick: one of logkeeper's rounds. +const MARGIN_MS: u64 = 5_000; + +/// The transfer so far: its bytes, when its first read came, every CPU's +/// counters then and how many bytes that read brought, and when its last came. +struct Progress { + bytes: u64, + first: Option<(Instant, Vec, u64)>, + last: Option, +} + +fn main() { + let bound_ms: u64 = std::env::var(toyos_tco::LIST_BOUND_ENV) + .unwrap_or_else(|e| panic!("{}: {e}; this job runs only in a job list", toyos_tco::LIST_BOUND_ENV)) + .parse() + .unwrap_or_else(|e| panic!("{}: {e}", toyos_tco::LIST_BOUND_ENV)); + let since_boot_ms = toyos_abi::clock::nanos_since_boot() / 1_000_000; + let window = Duration::from_millis(bound_ms.saturating_sub(MARGIN_MS).saturating_sub(since_boot_ms)); + let cap: SysCap = Endowments::get().take(SYSCAP_LABEL).expect("test-runner endows a capability"); + let progress = Mutex::new(Progress { bytes: 0, first: None, last: None }); + let rtt = Mutex::new(String::from("unread")); + // Dropped as the body ends whole, or as this panics. + let (transferring, ended) = mpsc::channel::<()>(); + std::thread::scope(|s| { + let (watched, rtt, cap) = (&progress, &rtt, &cap); + s.spawn(move || { + if ended.recv_timeout(window) == Err(mpsc::RecvTimeoutError::Timeout) { + let progress = watched.lock().unwrap_or_else(PoisonError::into_inner); + // The body ended whole while this waited for the lock. + if ended.try_recv() == Err(mpsc::TryRecvError::Disconnected) { + return; + } + let rtt = rtt.lock().unwrap_or_else(PoisonError::into_inner); + println!("https_download: {}", report("cut", None, &progress, &rtt, cap)); + std::process::exit(0); + } + }); + leased(); + *rtt.lock().unwrap_or_else(PoisonError::into_inner) = handshakes(); + + let url = format!("https://{HOST}{PATH}"); + let response = https_client::agent(ROOTS).get(&url).call().unwrap_or_else(|e| panic!("GET {url}: {e}")); + let mut body = response.into_body().into_reader(); + let mut digest = ring::digest::Context::new(&ring::digest::SHA256); + let mut chunk = vec![0u8; 64 * 1024]; + let mut bytes = 0u64; + loop { + let n = body.read(&mut chunk).unwrap_or_else(|e| panic!("the body of {url} after {bytes} bytes: {e}")); + if n == 0 { + break; + } + digest.update(&chunk[..n]); + bytes += n as u64; + let mut progress = progress.lock().unwrap_or_else(PoisonError::into_inner); + progress.bytes = bytes; + progress.last = Some(Instant::now()); + if progress.first.is_none() { + progress.first = Some((Instant::now(), round(&cap), bytes)); + } + } + let progress = progress.lock().unwrap_or_else(PoisonError::into_inner); + drop(transferring); + let rtt = rtt.lock().unwrap_or_else(PoisonError::into_inner); + let sha256 = https_client::hex(digest.finish()); + println!("https_download: {}", report("whole", Some(&sha256), &progress, &rtt, &cap)); + }); +} + +/// Wait for netstack to say it holds a lease; panics where it says it took +/// none. +fn leased() { + served_log::Log::open().until("netstack's word on its lease", LEASE_WAIT, |line| { + let Some(said) = program_line(line).filter(|said| said.tag == "netstack") else { return false }; + assert!(!said.text.starts_with(toyos_tco::NO_LEASE_SAID), "{}", said.text); + said.text.starts_with(toyos_tco::LEASE_SAID) + }); +} + +/// [`PROBES`] handshakes with [`HOST`]'s first address, each closed as it is +/// made, as the `rtt_ms` the module header gives. +fn handshakes() -> String { + let addr = (HOST, 443) + .to_socket_addrs() + .unwrap_or_else(|e| panic!("look up {HOST}: {e}")) + .next() + .unwrap_or_else(|| panic!("{HOST} has no address")); + let each: Vec = (0..PROBES) + .map(|_| { + let at = Instant::now(); + drop(TcpStream::connect(addr).unwrap_or_else(|e| panic!("connect to {HOST}: {e}"))); + format!("{:.1}", at.elapsed().as_secs_f64() * 1e3) + }) + .collect(); + format!("[{}]", each.join(" ")) +} + +/// Every CPU's counters now. +fn round(cap: &SysCap) -> Vec { + let mut raw = vec![RawRecord::EMPTY; syscall::cpu_count() as usize]; + let n = cap.counters(&mut raw).expect("test-runner's capability reads the counters"); + raw[..n].iter().map(|r| Record::decode(r).expect("a record that decodes")).collect() +} + +/// `word`, then the transfer from its first byte to now in the words the +/// module header gives them. +fn report(word: &str, sha256: Option<&str>, progress: &Progress, rtt: &str, cap: &SysCap) -> String { + let (Some((at, before, first)), Some(last)) = (&progress.first, progress.last) else { + return format!("{word} before the first byte of the body"); + }; + let after = round(cap); + let secs = last.duration_since(*at).as_secs_f64(); + let busy: Option> = before + .iter() + .zip(&after) + .map(|(a, b)| { + let ran = b.get(Counter::Mperf)? - a.get(Counter::Mperf)?; + let stamp = b.get(Counter::Stamp)? - a.get(Counter::Stamp)?; + Some(ran as f64 / stamp as f64) + }) + .collect(); + let busy = match busy { + Some(each) => format!( + "busy={:.3} cpus=[{}]", + each.iter().sum::() / each.len() as f64, + each.iter().map(|b| format!("{b:.3}")).collect::>().join(" ") + ), + None => "busy=unread".to_string(), + }; + let mbps = (progress.bytes - first) as f64 * 8.0 / secs / 1e6; + let sha256 = sha256.map(|hex| format!(" sha256={hex}")).unwrap_or_default(); + format!("{word} bytes={}{sha256} secs={secs:.3} mbps={mbps:.1} rtt_ms={rtt} {busy}", progress.bytes) +} diff --git a/tests/toyos-rust-tests/src/bin/https_get.rs b/tests/toyos-rust-tests/src/bin/https_get.rs index e72b78dde80..c09e30238e4 100644 --- a/tests/toyos-rust-tests/src/bin/https_get.rs +++ b/tests/toyos-rust-tests/src/bin/https_get.rs @@ -1,6 +1,5 @@ -//! An HTTPS client as any Rust program writes one: `ureq` on `rustls` on -//! `ring`, as published, trusting the authorities of one roots file and no -//! others, and naming itself with the project's `User-Agent`. +//! An HTTPS client as any Rust program writes one (`https_client`), reading +//! one URL. //! //! argv: the URL, then the roots file. It says one line and ends: //! `https_get: ok bytes= sha256=` and 0 for a body read to its end, @@ -10,25 +9,15 @@ use std::io::Read; use std::process::ExitCode; -use ureq::tls::{PemItem, RootCerts, TlsConfig}; - -/// Root `CLAUDE.md`'s: the only `User-Agent` ToyOS sends. -const USER_AGENT: &str = "toyos-build (https://github.com/ToyOSOrg/ToyOS)"; +#[path = "../https_client.rs"] +mod https_client; fn main() -> ExitCode { let args: Vec = std::env::args().skip(1).collect(); let [url, roots] = args.as_slice() else { panic!("usage: https_get , not {args:?}"); }; - let pem = std::fs::read(roots).unwrap_or_else(|e| panic!("read {roots}: {e}")); - let certs: Vec<_> = ureq::tls::parse_pem(&pem) - .map(|item| match item.unwrap_or_else(|e| panic!("{roots}: {e}")) { - PemItem::Certificate(cert) => cert, - _ => panic!("{roots} holds a PEM block that is not a certificate"), - }) - .collect(); - let tls = TlsConfig::builder().root_certs(RootCerts::new_with_certs(&certs)).build(); - let agent = ureq::Agent::config_builder().user_agent(USER_AGENT).tls_config(tls).build().new_agent(); + let agent = https_client::agent(roots); let response = match agent.get(url).call() { Ok(response) => response, @@ -57,8 +46,7 @@ fn main() -> ExitCode { digest.update(&chunk[..n]); bytes += n; } - let hex: String = digest.finish().as_ref().iter().map(|b| format!("{b:02x}")).collect(); - println!("https_get: ok bytes={bytes} sha256={hex}"); + println!("https_get: ok bytes={bytes} sha256={}", https_client::hex(digest.finish())); ExitCode::SUCCESS } diff --git a/tests/toyos-rust-tests/src/https_client.rs b/tests/toyos-rust-tests/src/https_client.rs new file mode 100644 index 00000000000..7c88bb2c226 --- /dev/null +++ b/tests/toyos-rust-tests/src/https_client.rs @@ -0,0 +1,27 @@ +//! An HTTPS client as any Rust program writes one: `ureq` on `rustls` on +//! `ring`, as published, trusting the authorities of one roots file and no +//! others, and naming itself with the project's `User-Agent`. + +use ureq::tls::{PemItem, RootCerts, TlsConfig}; + +/// Root `CLAUDE.md`'s: the only `User-Agent` ToyOS sends. +const USER_AGENT: &str = "toyos-build (https://github.com/ToyOSOrg/ToyOS)"; + +/// The agent that trusts the certificates of `roots`, a PEM file, and no +/// others. +pub fn agent(roots: &str) -> ureq::Agent { + let pem = std::fs::read(roots).unwrap_or_else(|e| panic!("read {roots}: {e}")); + let certs: Vec<_> = ureq::tls::parse_pem(&pem) + .map(|item| match item.unwrap_or_else(|e| panic!("{roots}: {e}")) { + PemItem::Certificate(cert) => cert, + _ => panic!("{roots} holds a PEM block that is not a certificate"), + }) + .collect(); + let tls = TlsConfig::builder().root_certs(RootCerts::new_with_certs(&certs)).build(); + ureq::Agent::config_builder().user_agent(USER_AGENT).tls_config(tls).build().new_agent() +} + +/// A digest as lower-case hex. +pub fn hex(digest: ring::digest::Digest) -> String { + digest.as_ref().iter().map(|b| format!("{b:02x}")).collect() +} diff --git a/tests/toyos.rs b/tests/toyos.rs index 9eb055bb85a..f44fdc5f86f 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -151,6 +151,9 @@ const RUST_SKIP: &[&str] = &[ // `https_fetch` runs it on the boot its fetches run on, so that test // carries its own oracle; a shared run would be the same QEMU CPU twice. "ring_kat", + // It downloads from the internet, inside a job list's bound alone: the + // `internet_download` metal row runs it. + "https_download", // It asserts nothing at all: it holds `dump_nmi_probe`'s boot open for // twenty seconds. On a shared boot it would be twenty seconds of nothing. "lan_hold", @@ -940,8 +943,53 @@ const METAL: &[(&str, metal::Metal)] = &[ }, }, ), + // ---- one image: tests/downloadcase, the internet download ---- + ( + // A public file over HTTPS through netstack on the I219, for as long as + // the list's bound leaves it: its rate is reported, never judged. A boot + // of its own, so the transfer has the machine alone. + "internet_download", + metal::Metal { + arms: &[metal::once("download", "tests/downloadcase", &[], &["test_rs_https_download"])], + judge: |b| https_download_on_metal(b[0]), + }, + ), ]; +/// What `https_download` said of its transfer, printed as it said it: green +/// for a job that ended 0 having read some of the body, and, where it read +/// all of it, a body of the file's length and hash. Its busy fraction is +/// read: the T14 counts MPERF on every CPU (`counters_on_metal`), so one it +/// could not read is the job's defect. +fn https_download_on_metal(back: &metal::Readback) -> Result<(), String> { + /// The job's file's length, and the SHA-256 its publisher states beside it + /// (`.sha256`). + const DOWNLOAD_BYTES: &str = "170439044"; + const DOWNLOAD_SHA256: &str = "1a9ee8caaa18a3e433fef93cea8a55dc1ebd478ed761b2fef69d4565f9d00e7f"; + const SAID: &str = "https_download: "; + let log = back.log(); + let said = log.text().lines().find_map(|line| line.find(SAID).map(|at| line[at + SAID.len()..].trim())); + if let Some(said) = said { + eprintln!(" [download] {said}"); + } + back.job_passed("test_rs_https_download")?; + let said = said.ok_or_else(|| format!("https_download exited 0 and said no line opening {SAID:?}"))?; + let field = |name: &str| said.split_whitespace().find_map(|word| word.strip_prefix(name)?.strip_prefix('=')); + match said.split_whitespace().next() { + Some("whole") if field("bytes") == Some(DOWNLOAD_BYTES) && field("sha256") == Some(DOWNLOAD_SHA256) => {} + Some("whole") => { + return Err(format!("https_download's body is not {DOWNLOAD_BYTES} bytes hashing {DOWNLOAD_SHA256}: {said}")) + } + // A cut says its bytes only where the body's first read came. + Some("cut") if field("bytes").is_some() => {} + _ => return Err(format!("https_download read none of the body: {said}")), + } + if field("busy") == Some("unread") { + return Err(format!("https_download read no MPERF on the T14: {said}")); + } + Ok(()) +} + /// A boot whose kernel leaves the i8042 unprobed, so the one grantable row is /// free: the ports' job first, since its last holder keeps them until it ends. const ISA_WITHHELD: &[metal::Arm] = &[metal::once( @@ -3265,13 +3313,12 @@ fn boot_netcase( rust_bins: &[(String, Vec)], options: BootOptions, ) -> Result { - const LEASED: &str = "netstack: DHCP: lease "; /// `userland/netstack/src/serve.rs`'s line for a probing no host answered. const CLAIMED: &str = "netstack: mDNS: no host answered for "; let case = compile::repo_root().join("tests/netcase"); let mut qemu = QemuInstance::boot_with_options(&case, c_bins, rust_bins, options); let mut console = qemu.boot_log().to_string(); - await_marker(&mut qemu, &mut console, LEASED, "netstack's lease").map_err(|e| format!("{e}\n{console}"))?; + await_marker(&mut qemu, &mut console, toyos_tco::LEASE_SAID, "netstack's lease").map_err(|e| format!("{e}\n{console}"))?; await_marker(&mut qemu, &mut console, CLAIMED, "netstack to claim its name").map_err(|e| format!("{e}\n{console}"))?; Ok(qemu) } diff --git a/toyos-tco/src/lib.rs b/toyos-tco/src/lib.rs index d0b6e9c0ab8..b5c039a7d72 100644 --- a/toyos-tco/src/lib.rs +++ b/toyos-tco/src/lib.rs @@ -126,11 +126,22 @@ pub const fn list_bound_ms(members_ms: u64) -> u64 { JOB_BOUND_MS + members_ms } +/// The environment variable the test runner sets on each job of a list to the +/// bound it gave that list, in milliseconds from boot: what a job that must +/// end its own work inside the list reads. +pub const LIST_BOUND_ENV: &str = "TOYOS_LIST_BOUND_MS"; + /// The bound netstack gives this machine's first DHCP lease before it says it has /// none and serves anyway, in milliseconds. The harness waits it out on a wire /// with no server, so the two read one declaration. pub const LEASE_BOUND_MS: u64 = 20_000; +/// What netstack's line on a lease it took opens with, and its line on +/// [`LEASE_BOUND_MS`] passing with none: netstack says them, and the harness +/// and the jobs that wait on its lease read them. +pub const LEASE_SAID: &str = "netstack: DHCP: lease "; +pub const NO_LEASE_SAID: &str = "netstack: DHCP: no lease as "; + /// The bound a panicked kernel holds its panel for before it returns the /// machine to firmware itself, in milliseconds. A minute is this project's /// bound for every watchdog. diff --git a/userland/netstack/src/main.rs b/userland/netstack/src/main.rs index 619aa93f2a1..d4afc0b97c8 100644 --- a/userland/netstack/src/main.rs +++ b/userland/netstack/src/main.rs @@ -207,8 +207,8 @@ impl Leases { // read off a stick or a stream has this line and nothing else // to say what this machine's network was. Some(lease) => say!( - "netstack: DHCP: lease {}/{} from {}, gateway {}, dns [{}], {} ms after \ - netstack came up", + "{}{}/{} from {}, gateway {}, dns [{}], {} ms after netstack came up", + toyos_tco::LEASE_SAID, lease.address, lease.prefix_len, lease.server, @@ -231,8 +231,8 @@ impl Leases { return false; } say!( - "netstack: DHCP: no lease as {} in {} s; this machine has no address and every \ - connect through it is refused", + "{}{} in {} s; this machine has no address and every connect through it is refused", + toyos_tco::NO_LEASE_SAID, HOSTNAME, self.began.elapsed().as_secs(), ); diff --git a/userland/test-runner/src/main.rs b/userland/test-runner/src/main.rs index 2efb1c5b9e8..2364fef168e 100644 --- a/userland/test-runner/src/main.rs +++ b/userland/test-runner/src/main.rs @@ -90,7 +90,7 @@ fn main() { } *RUNNING.lock().expect("the deadline thread does not panic holding this") = job.clone(); - match run_one(job, &[], cap.as_ref()) { + match run_one(job, &[], Some(bound_ms), cap.as_ref()) { Ran::No => { give_the_machine_back(&format!("job {job:?} did not run")) } @@ -212,7 +212,7 @@ fn command(line: &str, cap: Option<&SysCap>) { let mut words = name.split_whitespace(); let Some(name) = words.next() else { return }; let args: Vec<&str> = words.collect(); - run_one(name, &args, cap); + run_one(name, &args, None, cap); } enum Ran { @@ -221,8 +221,9 @@ enum Ran { Spawned, } -/// Run `/system/bin/` between the host's markers. -fn run_one(name: &str, args: &[&str], cap: Option<&SysCap>) -> Ran { +/// Run `/system/bin/` between the host's markers, telling it the bound +/// of the list it is in, where it is in one (`toyos_tco::LIST_BOUND_ENV`). +fn run_one(name: &str, args: &[&str], list_bound_ms: Option, cap: Option<&SysCap>) -> Ran { let path = format!("/system/bin/{name}"); println!("===TEST_START {name}==="); @@ -231,6 +232,9 @@ fn run_one(name: &str, args: &[&str], cap: Option<&SysCap>) -> Ran { // Piped stdin so the child does not consume the serial commands. let mut command = Command::new(&path); command.args(args).stdin(Stdio::piped()); + if let Some(ms) = list_bound_ms { + command.env(toyos_tco::LIST_BOUND_ENV, ms.to_string()); + } // **A refused dup is an answer and not a failure — but only one // refusal is.** `duplicate` needs `DUP` on the capability, which a // manifest grants by name, so `PermissionDenied` says this cap is one From 089c9b7130eae4f4bbdaf72fef8f2f328024f4f8 Mon Sep 17 00:00:00 2001 From: japabu Date: Fri, 9 Oct 2026 22:03:40 +0200 Subject: [PATCH 2/3] The window issue's exit is one only the fix can meet, and its records cite the T14 reading at 83ef5a9ae The receive-window issue's exit asked for a T14 rate above 524.3 over the smallest handshake time. Each handshake time bounds the round trip from above, so that figure bounds the 64 KiB window's ceiling from below, and the reading at 83ef5a9ae (35.3 Mb/s against 34.95) met it with the defect unchanged. The exit now asks for both a toyos-net-tcp host test that a stream's SYN carries a window shift above 0 and its empty-buffer window exceeds 65,535, and a T14 rate above twice that figure, which a 64 KiB window reaches only if the fastest handshake takes twice the round trip; the five read spread by 12 %. The issue's Measured section records that boot: rtt_ms [16.8 15.7 15.0 16.0 16.5], 35.3 Mb/s, busy 0.016. The network track's citations of b323e70ee, a commit amended into 83ef5a9ae and reachable from no branch, cite that boot; its log shows the lease line and the mDNS claim with no loss line. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C --- ...ion-at-one-64-kib-window-per-round-trip.md | 30 +++++++++++++------ issues/toyos-has-its-own-network-stack.md | 4 +-- 2 files changed, 23 insertions(+), 11 deletions(-) diff --git a/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md b/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md index 2d583243999..2500eea758e 100644 --- a/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md +++ b/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md @@ -21,13 +21,17 @@ netstack's places may hold. ## Measured -The T14, `internet_download` at `b323e70ee`, one boot, the link up at -1000 Mb/s: `whole bytes=170439044 secs=38.826 mbps=35.1 busy=0.014`. -35.1 Mb/s is 65,535 bytes every 14.9 ms, and the CPUs were 1.4 % busy across -the transfer. That boot did not read the connection's round trip, so the -window is the ceiling by arithmetic and not yet by measurement; the job says -`rtt_ms`, each of its handshakes' times, from the next boot on. netstack exposes no connection's SRTT: its `inspect` -snapshot counts sockets and nothing of one. +The T14, `internet_download` at `83ef5a9ae`, one boot, the link up at +1000 Mb/s: `whole bytes=170439044 secs=38.657 mbps=35.3 +rtt_ms=[16.8 15.7 15.0 16.0 16.5] busy=0.016`, judged `PASS`. One window per +round trip is 524.3 / `rtt_ms` Mb/s: 31.2 to 35.0 across the five handshakes, +and the rate sits at the top of that range with the CPUs 1.6 % busy across the +transfer. Each `rtt_ms` is one handshake, a round trip plus netstack's own time +for a connect, so it bounds the path's round trip from above and 524.3 over it +bounds the window's ceiling from below: a rate held at the ceiling clears that +bound by up to the handshakes' spread, 12 % in this boot. netstack exposes no +connection's SRTT: its `inspect` snapshot counts sockets and nothing of one. +The line's own rate is not measured. ## Owner @@ -35,5 +39,13 @@ snapshot counts sockets and nothing of one. ## Exit -A T14 `internet_download` boot whose `mbps` exceeds 524.3 over the smallest -of its `rtt_ms` (65,535 bytes a round trip, in Mb/s). +Both of: + +- A host test in `toyos-net-tcp`, at netstack's stream buffer size, that a + stream's SYN carries a window shift above 0 and that the window it + advertises with an empty receive buffer exceeds 65,535 bytes. A buffer of + 65,535 bytes fails both. +- A T14 `internet_download` boot whose `mbps` exceeds twice 524.3 over the + smallest of its `rtt_ms`: 69.9 Mb/s at 15.0 ms. A 64 KiB window reaches + it only if the fastest handshake takes twice the path's round trip, where + the five at `83ef5a9ae` spread by 12 %. diff --git a/issues/toyos-has-its-own-network-stack.md b/issues/toyos-has-its-own-network-stack.md index ec0820e6512..3eaa8589eb8 100644 --- a/issues/toyos-has-its-own-network-stack.md +++ b/issues/toyos-has-its-own-network-stack.md @@ -21,7 +21,7 @@ Stage 5 lands in slices, the orchestrator's cut under the owner's words "i want What the node does not yet meet: -- Of what a third party wrote, it has met QEMU's user network in a guest, on virtio and on QEMU's e1000e: its DHCP server, its ARP and the host kernel's TCP behind it (`netstack_streams`, `netstack_socket_churn`, `libc_sockets`). On the T14 it has met one router, its DHCP server and a CDN behind it: `internet_download` green at `b323e70ee`, one boot. Exit: the T14's outbound rows are green on it. +- Of what a third party wrote, it has met QEMU's user network in a guest, on virtio and on QEMU's e1000e: its DHCP server, its ARP and the host kernel's TCP behind it (`netstack_streams`, `netstack_socket_churn`, `libc_sockets`). On the T14 it has met one router, its DHCP server and a CDN behind it: `internet_download` green at `83ef5a9ae`, one boot: the whole body, its hash the pin, 35.3 Mb/s. Exit: the T14's outbound rows are green on it. - A connection's receive window is at most 65,535 bytes, so a download takes at most that per round trip: `issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md`. - It counts a DHCP message [udp] refused as `node.dhcp-unsent`, whatever the rule; the `dhcp.renew-unroutable` scenario owed above is not written. Exit: that scenario names the counter, or the node counts the renewal apart. - With the link down the DHCP client keeps its timers: every 4 to 64 s the node has a deadline, builds a DISCOVER that [udp] refuses, and counts it in `dhcp.tx.discover` and `node.dhcp-unsent`, from `Node::new` on. Exit: `toyos-dhcp`'s client is told the link went down and waits for it, and the node's first DISCOVER is the one that leaves. @@ -39,7 +39,7 @@ What the node does not yet meet: - The word each of [udp]'s refusals is answered in on the pipe ABI is `Refused` in `userland/netstack/node/src/datagram.rs`, tested by `each_refusal_is_answered_in_the_pipes_word_for_it` against no reader's scenario; `udp.no-ephemeral-port` has a word and no test that reaches it. Exit: the scenario owed below exists, and the test names its id. - A client's datagram to a broadcast address needs its socket's permission, where smoltcp sends it for every socket. The owner's ruling: "Of course carry the permission why would you even ask. The premise is and always has been to do things properly". The pipe ABI carries the permission as far as netd: `MsgType::UdpSetOption` with `OPT_BROADCAST` (`toyos/src/net.rs`), sent by std's `set_broadcast` (`sdk/std/sys/net/connection.rs`) and by libc's `setsockopt` of `SO_BROADCAST` (`userland/libc/src/socket.rs`), which keeps it for a socket not yet bound and sends it at `bind`; the word for a send refused for want of it is `ERR_PERMISSION_DENIED`, `ErrorKind::PermissionDenied` in std and `EACCES` in libc. The node enforces it, and netstack hands it the request (`Sockets::udp_set_option`, `userland/netstack/src/serve.rs`): `Node::udp_set_broadcast` hands the permission to `Udp::set_broadcast`, and a send without it is refused `udp.broadcast-not-permitted` and answered `Refused::PermissionDenied` (`userland/netstack/node/src/datagram.rs`), which `a_broadcast_needs_its_permission` (US-21, `userland/netstack/node/tests/datagram.rs`) holds, with both broadcast addresses read off the wire in link-broadcast frames once the socket holds it and refused again once it is taken back, and `each_refusal_is_answered_in_the_pipes_word_for_it` holds the word. The permission of the call goes with the datagram to the route it leaves by: [udp] queues it with what its socket held, a closed socket's included, and `Ip::send_udp` refuses a link broadcast to one without it, `ip.broadcast-not-permitted`, counted and logged, so a prefix a DHCP server renews under a held address between the call and the frame makes no broadcast of a datagram accepted for a host (`s_udp_us_021_shard_a_datagram_accepted_for_a_host_is_no_broadcast_after_a_renewal_narrows_the_prefix` and its two neighbours in `toyos-net-shard/tests/udp.rs`, and from a server's ACK `a_datagram_accepted_for_a_host_is_no_broadcast_after_the_server_renews_a_narrower_prefix` in the node's tests). The search is run. `grep -rnI -E 'SO_BROADCAST|INADDR_BROADCAST|255\.255\.255\.255|sendto|setsockopt|sys/socket\.h|socket\(' userland/doom/doomgeneric`, the one third-party C program an image builds, at the commit `userland/doom/build.rs` pins, exits 1 with no line; the same without `sys/socket\.h` over `tests/testcases` exits 1 with no line; `SO_BROADCAST|INADDR_BROADCAST|255\.255\.255\.255` over the trees of `src/libcxx.rs`'s `SOURCES` finds LLVM libc's own macro and its documentation and no caller; and in ToyOS's tree outside the stack's crates the option is named only by the two setters. The socket2 fork's setter reaches nobody: `issues/the-socket2-forks-so-broadcast-setter-does-nothing.md`. Exit: a guest test, `udp_broadcast_needs_its_permission`, reads `broadcast()` false, then true after `set_broadcast(true)`, on the socket and on a `try_clone` of it, which `broadcast()` answering a constant or a duplicate holding a value of its own turns red, and sends to the limited broadcast address before and after the set and is refused `PermissionDenied` only before, which std's setter reverted to `Ok(())` turns red; and a guest C case reads libc's setter, `socket`, `setsockopt(SO_BROADCAST)`, `bind`, `sendto` to the limited broadcast address, refused `EACCES` without the option and sent with it, and with the option set before `bind`, which `bind` no longer handing the kept option over turns red. The case runs the sequence without `bind` too, where the first `sendto` binds the socket and hands the kept option over; `tests/netcase/sendto_unbound.c` holds that send for a socket permitted to make it. - The machine's name is told to the node twice, as `toyos_dhcp::HostName` in `Node::new` and as `toyos_mdns::Host` in `Node::answer_as`: the first owns its bytes and the second borrows them. netstack's is a constant, so neither takes a leak. Exit: one type carries the label to both. -- The machine's name is probed for and defended (RFC 6762 §8, §9) by `toyos-mdns` (`userland/netstack/mdns`), on the node, which netstack runs. What that does not yet meet. Every machine is named `toyos-t14` (`HOSTNAME`, `userland/netstack/src/main.rs`) and a responder that loses its name holds none, so of two on one network the second answers to no name: asked "When two machines on one network have the same ToyOS host name, what should the second one do?", the owner answered "Hold no name, say so (Recommended)". Asked on 2026-10-09 "A ToyOS machine that loses its .local name to a conflict stays nameless until its network link next returns, even after the other machine has left. Should it try for its name again by itself?", he answered "Retry on a slow interval (Recommended)". Built: a lost name is probed for again a minute after each loss (`RETRY_MS`, `userland/netstack/mdns/src/lib.rs`) and at once on a link after none, and is claimed by the first probing no host answers; RFC 6762 permits probing again at §8.1's rate ("wait five seconds after any failed probe attempt before trying again") and does not require it. Two packets from any host on the link still take a held name, a response to take it back to probing and a response under the probe, and one more a minute keeps it: no path back exists that a peer cannot hold shut, since a host that answers every probe is what a holder of the name is. The caller does not say how a message was addressed, so two rules that turn on it are not applied: a response is read however it came, where §6 reads a unicast one only within two seconds of a question that asked for one; and a message from a source outside the subnet is ignored even when it was sent to the group, where §11 deems that on the link "regardless of source IP address", so a host of the same name on another subnet of the link is never a conflict. Exit: each machine's name is its own, and a test with two guests on one network finds each by its name; and the caller says how a message was addressed, with a test in which a unicast response nobody asked for takes no name and one in which a response sent to the group from another subnet does. The shipped responder's claim on the T14's wired card is read once: the `internet_download` boot at `b323e70ee` shows the lease line, then `netstack: mDNS: no host answered for toyos-t14.local; this machine answers as it`, and no loss line. Not measured, with its exit: the link's return on metal (the orchestrator's attended session with the owner, never automated). A link that goes and returns inside one pass of the Intel driver reaches the name as no change: `issues/a-link-that-goes-and-returns-inside-one-pass-of-the-intel-driver-is-reported-as-no-change.md`. +- The machine's name is probed for and defended (RFC 6762 §8, §9) by `toyos-mdns` (`userland/netstack/mdns`), on the node, which netstack runs. What that does not yet meet. Every machine is named `toyos-t14` (`HOSTNAME`, `userland/netstack/src/main.rs`) and a responder that loses its name holds none, so of two on one network the second answers to no name: asked "When two machines on one network have the same ToyOS host name, what should the second one do?", the owner answered "Hold no name, say so (Recommended)". Asked on 2026-10-09 "A ToyOS machine that loses its .local name to a conflict stays nameless until its network link next returns, even after the other machine has left. Should it try for its name again by itself?", he answered "Retry on a slow interval (Recommended)". Built: a lost name is probed for again a minute after each loss (`RETRY_MS`, `userland/netstack/mdns/src/lib.rs`) and at once on a link after none, and is claimed by the first probing no host answers; RFC 6762 permits probing again at §8.1's rate ("wait five seconds after any failed probe attempt before trying again") and does not require it. Two packets from any host on the link still take a held name, a response to take it back to probing and a response under the probe, and one more a minute keeps it: no path back exists that a peer cannot hold shut, since a host that answers every probe is what a holder of the name is. The caller does not say how a message was addressed, so two rules that turn on it are not applied: a response is read however it came, where §6 reads a unicast one only within two seconds of a question that asked for one; and a message from a source outside the subnet is ignored even when it was sent to the group, where §11 deems that on the link "regardless of source IP address", so a host of the same name on another subnet of the link is never a conflict. Exit: each machine's name is its own, and a test with two guests on one network finds each by its name; and the caller says how a message was addressed, with a test in which a unicast response nobody asked for takes no name and one in which a response sent to the group from another subnet does. The shipped responder's claim on the T14's wired card is read once: the `internet_download` boot at `83ef5a9ae` shows the lease line, then `netstack: mDNS: no host answered for toyos-t14.local; this machine answers as it`, and no loss line. Not measured, with its exit: the link's return on metal (the orchestrator's attended session with the owner, never automated). A link that goes and returns inside one pass of the Intel driver reaches the name as no change: `issues/a-link-that-goes-and-returns-inside-one-pass-of-the-intel-driver-is-reported-as-no-change.md`. - A socket's datagrams that wait for a next hop wait in the one queue it has, `toyos_net_udp::limits::TX_DATAGRAMS` of them: with all 16 waiting for one next hop, the socket's sends to every other destination are refused `udp.tx-queue-full`, `Refused::ResourceExhausted` to a client, until that hop answers or [ip] gives it up, 3 s after its first request left (`BROADCAST_SOLICIT` requests a `RETRANS` apart). It is no regression by this track's earlier record of what ships, that smoltcp kept each datagram in its own socket until the neighbour answered. What a closed socket had accepted waits in the closed sender's `CLOSED_DATAGRAMS` the same way, so 16 datagrams of closed sockets that wait for one silent host have the next closes' datagrams discarded, `udp.tx-discarded-on-close`, for those 3 s: the line on US-53 among stage 3's departures. `s_udp_us_025_what_waits_for_a_next_hop_is_bounded_by_its_sockets_queue` holds the first as it stands. The move's guest runs showed no program refused for it: none of their jobs sends sixteen datagrams past a hop that does not answer. Owner: whoever reads the T14's first outbound run on this stack. Exit: a run on the T14 or in a guest shows a program refused for it, and a socket's bound is counted per next hop; or the owner rules the one queue is the bound. - A link that goes down drops no waiting datagram by itself: every waiting one is handed to [ip] again at the next transmit opportunity, which refuses it for want of a route (`route.no-source-address`), and one for which the link is back before that opportunity waits for its next hop anew, where the readers' link-down rule (NUD-23) drops what waits at once. `s_udp_us_024_shard_a_waiting_datagram_whose_route_goes_is_dropped` holds the first as it stands, and `a_waiting_datagram_whose_link_returns_before_an_opportunity_waits_anew_and_leaves`, beside it in `toyos-net-shard/tests/udp.rs` and under no scenario's id, the second. Not done before the move, whose worker changed nothing below netstack. Owner: whoever next changes `toyos-net-udp`. Exit: that second test's last assertion is inverted and finds the datagram that waited gone, or the specifications have it wait. - `toyos_dns::Lookup::on_datagram` still takes a reply's source address and port, which the node's connected sockets have already matched and the node hands it from its own record of the query: the resolver that read them left with smoltcp, and the node hands both from its own record. Not removed at the move, which left `toyos-dns` as it was. Exit: the two parameters and the comparison inside `on_datagram` go. From 3b196d0754754ba8d786aeca13d531ba9ac332e5 Mon Sep 17 00:00:00 2001 From: japabu Date: Fri, 9 Oct 2026 22:10:20 +0200 Subject: [PATCH 3/3] The window issue records the T14's gigabit line by the owner's word, and its T14 exit is a reading after the fix with no rate bar The review's note: the issue's T14 arm assumed a line rate nobody had measured. The owner, asked the line's speed, answered "No bar but its a gigabit connection and i would like close to that". Measured now records that the T14 sits on a gigabit connection wired to its router, by the owner's statement, and that the goal is close to it. The exit keeps the toyos-net-tcp host-test arm; its T14 arm becomes a recorded internet_download reading after the fix, with no rate threshold. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C --- ...onnection-at-one-64-kib-window-per-round-trip.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md b/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md index 2500eea758e..b78ba095472 100644 --- a/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md +++ b/issues/netstacks-receive-buffer-caps-every-connection-at-one-64-kib-window-per-round-trip.md @@ -31,7 +31,11 @@ for a connect, so it bounds the path's round trip from above and 524.3 over it bounds the window's ceiling from below: a rate held at the ceiling clears that bound by up to the handshakes' spread, 12 % in this boot. netstack exposes no connection's SRTT: its `inspect` snapshot counts sockets and nothing of one. -The line's own rate is not measured. + +The T14 sits on a gigabit connection, wired to its router, by the owner's +statement; the line's own rate is not measured. The goal is close to that: +"No bar but its a gigabit connection and i would like close to that" (the +owner, asked the line's speed). ## Owner @@ -45,7 +49,6 @@ Both of: stream's SYN carries a window shift above 0 and that the window it advertises with an empty receive buffer exceeds 65,535 bytes. A buffer of 65,535 bytes fails both. -- A T14 `internet_download` boot whose `mbps` exceeds twice 524.3 over the - smallest of its `rtt_ms`: 69.9 Mb/s at 15.0 ms. A 64 KiB window reaches - it only if the fastest handshake takes twice the path's round trip, where - the five at `83ef5a9ae` spread by 12 %. +- A T14 `internet_download` reading after the fix, recorded here with its + `mbps` and `rtt_ms`. It carries no rate threshold: the owner set "No bar but + its a gigabit connection and i would like close to that".