From 2966a309a6a939674ca710cf732aafc6c81254ec Mon Sep 17 00:00:00 2001 From: japabu Date: Sun, 11 Oct 2026 23:39:15 +0200 Subject: [PATCH 01/14] netring: the design round opens The session-region design that replaces the pipes between netstack and its clients, and its roast, are this pull request's body. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C From 148c3de6ea6912cf78a55fcf86060729154bff4a Mon Sep 17 00:00:00 2001 From: japabu Date: Sun, 11 Oct 2026 23:41:14 +0200 Subject: [PATCH 02/14] issues: netstack's pipe copies are a track, and a pipe is a 2 MiB page for a mapping nothing makes The track records the copies and the lock as a present-state weakness, with round 3's T14 rows as evidence and the stages the design in PR #863's body names. The defect is the pipe-section finding of that design: SYS_PIPE_MAP has had no caller outside the guest tests since a23734684, and it is why every pipe and every connection direction costs a 2 MiB page. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C --- ...bibyte-page-for-a-mapping-nothing-makes.md | 26 ++++++++ ...ves-every-byte-through-two-kernel-pipes.md | 60 +++++++++++++++++++ 2 files changed, 86 insertions(+) create mode 100644 issues/a-pipe-is-a-two-mebibyte-page-for-a-mapping-nothing-makes.md create mode 100644 issues/netstack-moves-every-byte-through-two-kernel-pipes.md diff --git a/issues/a-pipe-is-a-two-mebibyte-page-for-a-mapping-nothing-makes.md b/issues/a-pipe-is-a-two-mebibyte-page-for-a-mapping-nothing-makes.md new file mode 100644 index 0000000000..dae27a84dd --- /dev/null +++ b/issues/a-pipe-is-a-two-mebibyte-page-for-a-mapping-nothing-makes.md @@ -0,0 +1,26 @@ +--- +status: open +kind: defect +opened: 2026-10-11 +--- + +# A pipe is a 2 MiB page for a mapping nothing makes + +A pipe's ring is one `pmm::alloc_page` page, `PIPE_SIZE = PAGE_2M` +(`kernel/src/pipe.rs`), allocated and zeroed on its first write, because +`SYS_PIPE_MAP` maps that page into a process and a 2 MiB page is the only one +this kernel maps. Nothing outside the guest tests calls `SYS_PIPE_MAP`: its +last caller, netd, mapped its clients' rings only to read the closed flags, +and stopped in a23734684 when they were found forgeable. Since then the map +costs every pipe and every IPC connection, which is two pipes, 2 MiB a +direction once a byte moves, whatever the traffic: a wake bell holds 2 MiB to +carry one byte, and so does each direction of a connection that carries one +request. It also keeps `Pipe::publish_ends` writing reader and writer flags +into the page for a reader that no longer exists, and the ring's header +contract over a page a process may write (`sdk/abi/src/ring.rs`). + +Exit: `SYS_PIPE_MAP`, `publish_ends`, the ring's closed flags and the guest +tests that exercise only the map are gone, a pipe's ring is sized for a byte +stream rather than for a mapping, and a guest test that opens a connection, +sends one request and reads `memory_total` less free memory finds no 2 MiB +moved. diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md new file mode 100644 index 0000000000..4c783670fc --- /dev/null +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -0,0 +1,60 @@ +--- +status: open +kind: track +opened: 2026-10-11 +--- + +# netstack moves every byte through two kernel pipes + +A received byte is copied three times between the card's buffer and the +application's: netstack writes it into its connection's heap ring +(`system/netstack/shard/tcp/src/ring.rs`), then into the client's pipe with a +`SYS_WRITE` whose copy runs under the one global `PIPES` lock with preemption +off (`kernel/src/pipe.rs`, `issues/the-global-pipe-lock-spans-a-user-copy.md`), +and the client's `SYS_READ` copies it out under the same lock. A sent byte is +copied into the pipe, out of it into netstack's stack chunk, into the +connection's send ring, and into the frame. Both buffers hold the same bytes: a +stream holds its pipes' 2 MiB pages beside its own 4 MiB rings +(`PLACE_BYTES`, `system/netstack/src/main.rs`). + +Evidence, PR #861's round 3 on the T14, stage B at K = 32, medians of six runs +per MB of a 170 MB HTTPS download: the pipe table's lock held 0.281 ms and +spun for 0.047 ms (`pipe_lock_hold_cycles`, `pipe_lock_spin_cycles`, TSC +time), netstack's ring write took 0.225 ms (`cy_ring_write`), against 5.447 ms +of C0. The kernel copied 2.00 bytes of pipe per byte downloaded +(`pipe_copy_bytes`). + +What a pipe also cannot say is carried by workarounds this track deletes: how +a stream ended is the order its two pipes close in, one bit +(`issues/a-streams-failure-reaches-its-client-as-a-reset-whatever-it-was.md`); +a stream's client leaving is learned only through those pipes +(`OWNERLESS_LIFE`, `system/netstack/node/src/streams.rs`); a datagram costs a +pipe write and a request on a fresh connection. + +The owner ruled the replacement a go, in his words "pipes aren't meant for a +lot of data transfer, especially not network", and dropped zero copy from the +NIC into an application's memory. The design and its roast are PR #863's +body. + +- Stage 0: the scout boot on the T14 that splits the lock's hold into its two + copies (PR #863's body names the patch and the rows). +- Stage 1: the kernel answers what a received handle is and what a lent + region is, its length and whether it is memory the kernel allocated + (`issues/a-received-handle-has-no-knowable-type.md`, + `issues/a-file-server-maps-a-lent-window-at-the-size-it-expects.md`). +- Stage 2: a byte ring with untrusted cursors and wake words in + `toyos-transport` (`lib/blockring/transport`), host-tested and modelled in + loom. +- Stage 3: a TCP stream is one region and one connection: netstack's receive + and send buffers are the region's two rings, the connection carries the + connect and then only doorbells and the hang-up; std, libc and the mio and + socket2 forks read and write the rings through `toyos::net`. +- Stage 4: a listener and a datagram socket the same way, and the pipe ABI's + messages, `SYS_CONNECTION_JOIN` and `toyos::net`'s pipe handles go. + +Owner: the orchestrator, who stages it. + +Exit: no byte of a client's stream or datagram crosses a kernel pipe +(`rg -n 'Piped|DATA_HANDLES|connection_join' sdk system` is empty), and a T14 +boot of the round 3 download on the session build reads `pipe_copy_bytes` +under 0.01 per MB downloaded beside the scout's rows. From aecdf5e50c119662289ac20dd806f4addafd6f22 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 11:19:47 +0200 Subject: [PATCH 03/14] issues: the netring track's stages carry their pull requests and the design as amended Stage 0 is done: the scout's reading on #863 (S = 0.192 ms per MB, the copies 73% of the hold) joins the evidence. The owner's acceptance and its constraint, that the kernel gains only what it alone can answer, is the design line every stage keeps (https://github.com/ToyOSOrg/ToyOS/pull/863#issuecomment-6114483274). Stage 1 is #867, narrowed by the orchestrator's ruling there (https://github.com/ToyOSOrg/ToyOS/pull/867#issuecomment-6120212295): the kernel's one new answer is fstat's DeviceMemory, SYS_HANDLE_RECV answers no kind and SYS_SHM_MAP no length, and a receiver judges a lent region through fstat before it maps it. It closes the lent-window issue alone; issues/a-received-handle-has-no-knowable-type.md stays open, narrowed by #867. Stage 2 is #878. The exit read pipe_copy_bytes "per MB downloaded", a unit the row does not print: the row is per byte downloaded, 2.00 today. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...ves-every-byte-through-two-kernel-pipes.md | 28 +++++++++++-------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 4c783670fc..f58fd01c13 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -22,7 +22,10 @@ per MB of a 170 MB HTTPS download: the pipe table's lock held 0.281 ms and spun for 0.047 ms (`pipe_lock_hold_cycles`, `pipe_lock_spin_cycles`, TSC time), netstack's ring write took 0.225 ms (`cy_ring_write`), against 5.447 ms of C0. The kernel copied 2.00 bytes of pipe per byte downloaded -(`pipe_copy_bytes`). +(`pipe_copy_bytes`). Stage 0's scout split the hold on one T14 boot: the two +copies are 73% of it, and what the session design removes on the receive path +reads 0.192 ms per MB at most +(https://github.com/ToyOSOrg/ToyOS/pull/863#issuecomment-6114364774). What a pipe also cannot say is carried by workarounds this track deletes: how a stream ended is the order its two pipes close in, one bit @@ -33,16 +36,19 @@ pipe write and a request on a fresh connection. The owner ruled the replacement a go, in his words "pipes aren't meant for a lot of data transfer, especially not network", and dropped zero copy from the -NIC into an application's memory. The design and its roast are PR #863's -body. - -- Stage 0: the scout boot on the T14 that splits the lock's hold into its two - copies (PR #863's body names the patch and the rows). -- Stage 1: the kernel answers what a received handle is and what a lent - region is, its length and whether it is memory the kernel allocated - (`issues/a-received-handle-has-no-knowable-type.md`, +NIC into an application's memory. He accepted the design on the scout's +reading with one constraint on every stage: the data path is userland's, and +the kernel gains only what it alone can answer +(https://github.com/ToyOSOrg/ToyOS/pull/863#issuecomment-6114483274). The +design and its roast are PR #863's body. + +- Stage 0, done: the scout boot on the T14 that splits the lock's hold into + its two copies (PR #863). +- Stage 1, PR #867: the kernel's one new answer is `SYS_FSTAT`'s + `DeviceMemory`, and a receiver judges a lent region through `fstat` before + it maps it (closes `issues/a-file-server-maps-a-lent-window-at-the-size-it-expects.md`). -- Stage 2: a byte ring with untrusted cursors and wake words in +- Stage 2, PR #878: a byte ring with untrusted cursors and want words in `toyos-transport` (`lib/blockring/transport`), host-tested and modelled in loom. - Stage 3: a TCP stream is one region and one connection: netstack's receive @@ -57,4 +63,4 @@ Owner: the orchestrator, who stages it. Exit: no byte of a client's stream or datagram crosses a kernel pipe (`rg -n 'Piped|DATA_HANDLES|connection_join' sdk system` is empty), and a T14 boot of the round 3 download on the session build reads `pipe_copy_bytes` -under 0.01 per MB downloaded beside the scout's rows. +under 0.01 per byte downloaded beside the scout's rows. From a917480996d6daf87a165cbeefbf1d595d1841d2 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 12:08:43 +0200 Subject: [PATCH 04/14] issues: the netring track's stage 3 has an exit and its own doorbell for send room, and the pipe page has an owner Review https://github.com/ToyOSOrg/ToyOS/pull/863#issuecomment-6121859611. The owner ruled zero copy out unless the design showed a useful benefit; the track said he dropped it. It now says the design dropped it under his condition, on its own reading of a 4% gain (BLOCKER 1). Stage 3 has its own exit (BLOCKER 5): a loom model of one stream's reader and writer asleep at once, and the T14 boot of round 3's download. That boot reads pipe_copy_bytes, which no landing tree has: round 3's b245592a6 added it, and the boot carries the scout's measurement-only commits onto stage 3's head (BLOCKER 4). The track's own exit reads stage 3's and the pipe ABI gone, `pipe_pair` in toyos::net included, so a datagram socket cannot keep its pipes under another name (NOTE 3). Stage 3 rings a writer waiting for send room on a pipe of the stream's own, the amendment PR #863's body puts to the orchestrator: one netstack-to-client doorbell for both wants lets a reader drain the writer's wake, and a TcpStream split across two threads hangs (BLOCKER 6). A scratch loom model over #878's stream ends at 7ea0ff9d6 deadlocks with one doorbell and passes with two. The scout's S bounds the lock's share the design removes, not the receive path's saving: the pipe syscalls' work outside the hold goes too. The pipe-page defect has an owner, the orchestrator who stages the track that pays its pages (BLOCKER 2), and an exit two searches read (BLOCKER 3): the used-memory guest test it named reads a machine-wide figure every demand fault and heap growth moves. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...bibyte-page-for-a-mapping-nothing-makes.md | 12 ++--- ...ves-every-byte-through-two-kernel-pipes.md | 45 ++++++++++++------- 2 files changed, 37 insertions(+), 20 deletions(-) diff --git a/issues/a-pipe-is-a-two-mebibyte-page-for-a-mapping-nothing-makes.md b/issues/a-pipe-is-a-two-mebibyte-page-for-a-mapping-nothing-makes.md index dae27a84dd..3412e6ad96 100644 --- a/issues/a-pipe-is-a-two-mebibyte-page-for-a-mapping-nothing-makes.md +++ b/issues/a-pipe-is-a-two-mebibyte-page-for-a-mapping-nothing-makes.md @@ -19,8 +19,10 @@ request. It also keeps `Pipe::publish_ends` writing reader and writer flags into the page for a reader that no longer exists, and the ring's header contract over a page a process may write (`sdk/abi/src/ring.rs`). -Exit: `SYS_PIPE_MAP`, `publish_ends`, the ring's closed flags and the guest -tests that exercise only the map are gone, a pipe's ring is sized for a byte -stream rather than for a mapping, and a guest test that opens a connection, -sends one request and reads `memory_total` less free memory finds no 2 MiB -moved. +Owner: the orchestrator, who stages +`issues/netstack-moves-every-byte-through-two-kernel-pipes.md`, whose streams +ring their doorbells through pipes and pay a page for each until this closes. + +Exit: `rg -n -i 'pipe_?map|publish_ends|RING_(WRITER|READER)_CLOSED' kernel +sdk tests` is empty, and a pipe's ring is no `pmm::alloc_page` page: +`rg -n 'alloc_page|PAGE_2M' kernel/src/pipe.rs` is empty. diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index f58fd01c13..793b870876 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -23,8 +23,9 @@ spun for 0.047 ms (`pipe_lock_hold_cycles`, `pipe_lock_spin_cycles`, TSC time), netstack's ring write took 0.225 ms (`cy_ring_write`), against 5.447 ms of C0. The kernel copied 2.00 bytes of pipe per byte downloaded (`pipe_copy_bytes`). Stage 0's scout split the hold on one T14 boot: the two -copies are 73% of it, and what the session design removes on the receive path -reads 0.192 ms per MB at most +copies are 73% of it, and of the lock the session design removes at most +0.192 ms per MB on the receive path, beside the pipe syscalls' work outside +the hold, which no row reads apart (https://github.com/ToyOSOrg/ToyOS/pull/863#issuecomment-6114364774). What a pipe also cannot say is carried by workarounds this track deletes: how @@ -35,12 +36,15 @@ a stream's client leaving is learned only through those pipes pipe write and a request on a fresh connection. The owner ruled the replacement a go, in his words "pipes aren't meant for a -lot of data transfer, especially not network", and dropped zero copy from the -NIC into an application's memory. He accepted the design on the scout's -reading with one constraint on every stage: the data path is userland's, and -the kernel gains only what it alone can answer -(https://github.com/ToyOSOrg/ToyOS/pull/863#issuecomment-6114483274). The -design and its roast are PR #863's body. +lot of data transfer, especially not network", and ruled zero copy from the +NIC into an application's memory out unless the design showed a useful +benefit. The design dropped it under that condition, on its own reading: the +one copy it would remove is 4% of C0, against a filter and a queue per flow, +header split, an IOMMU window per client and a reassembly that copies. He +accepted the design on the scout's reading with one constraint on every +stage: the data path is userland's, and the kernel gains only what it alone +can answer (https://github.com/ToyOSOrg/ToyOS/pull/863#issuecomment-6114483274). +The design and its roast are PR #863's body. - Stage 0, done: the scout boot on the T14 that splits the lock's hold into its two copies (PR #863). @@ -52,15 +56,26 @@ design and its roast are PR #863's body. `toyos-transport` (`lib/blockring/transport`), host-tested and modelled in loom. - Stage 3: a TCP stream is one region and one connection: netstack's receive - and send buffers are the region's two rings, the connection carries the - connect and then only doorbells and the hang-up; std, libc and the mio and - socket2 forks read and write the rings through `toyos::net`. + and send buffers are the region's two rings, and the connection carries the + connect, then only doorbells and the hang-up. netstack rings a reader on the + connection and a writer waiting for send room on a pipe of the stream's own, + so neither drains a wake rung for the other. std, libc and the mio and + socket2 forks read and write the rings through `toyos::net`. Exit: a loom + model of one stream's reader and writer, each asleep on its own want while + netstack's thread releases room and publishes data in either order, ends + with neither waiting, and reds with both wants rung on one doorbell; and a + T14 boot of round 3's `internet_download` on stage 3's head reads, in every + run, `pipe_copy_bytes` under 0.01 per byte downloaded, its C0, hold and spin + per MB posted beside the scout's rows. No tree that lands has that counter, + which round 3's b245592a6 added: the boot carries the scout's instrument, + the `Measurement only, never lands` commits of `origin/main..2f9e090b8` + (`wt/toyos-netring-scout`), onto stage 3's head, less the netstack stamps + on the pipe writes stage 3 deletes. - Stage 4: a listener and a datagram socket the same way, and the pipe ABI's messages, `SYS_CONNECTION_JOIN` and `toyos::net`'s pipe handles go. Owner: the orchestrator, who stages it. -Exit: no byte of a client's stream or datagram crosses a kernel pipe -(`rg -n 'Piped|DATA_HANDLES|connection_join' sdk system` is empty), and a T14 -boot of the round 3 download on the session build reads `pipe_copy_bytes` -under 0.01 per byte downloaded beside the scout's rows. +Exit: stage 3's exit, and the pipe ABI gone with stage 4: +`rg -n 'Piped|DATA_HANDLES|NOTIFY_HANDLES|connection_join|CONNECTION_JOIN' sdk +system kernel` and `rg -n 'pipe_pair' sdk/toyos/src/net.rs` are empty. From 0d166351a5503db2f2fd90c0a1997f31628367c3 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 15:21:33 +0200 Subject: [PATCH 05/14] issues: the netring track's stage 3 passes a take on within a direction, and its exit names the instrument whole Review round 2 of #863 (BLOCKER 1): a `poll` beside a blocking call of the same direction lost its wake. The kernel answers a pipe watch by the pipe's level when the poll's submitter looks, so the blocking holder's wait took netstack's one ring and the poll re-armed over a ring holding bytes or its end. Amendment B, put to the orchestrator in the pull request's body: no end is held across a wait, every waiter of a direction is counted under its end, a thread that took the doorbell with another counted re-arms the want and rings netstack if the direction is ready, and netstack, rung, rings a set client want whose direction is ready. A scratch loom model over #878's ends reads it: the rule as written reds in all three cases, amendment B passes, and B less the pass and B under want-relaxed red. Stage 3's exit now names two models that drive the client end and the ringing code it lands (NOTE 2), and the instrument as one diff over its base, which carries the download row as well as the counter (NOTE 3). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...ves-every-byte-through-two-kernel-pipes.md | 32 ++++++++++++------- 1 file changed, 20 insertions(+), 12 deletions(-) diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 793b870876..666f592403 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -59,18 +59,26 @@ The design and its roast are PR #863's body. and send buffers are the region's two rings, and the connection carries the connect, then only doorbells and the hang-up. netstack rings a reader on the connection and a writer waiting for send room on a pipe of the stream's own, - so neither drains a wake rung for the other. std, libc and the mio and - socket2 forks read and write the rings through `toyos::net`. Exit: a loom - model of one stream's reader and writer, each asleep on its own want while - netstack's thread releases room and publishes data in either order, ends - with neither waiting, and reds with both wants rung on one doorbell; and a - T14 boot of round 3's `internet_download` on stage 3's head reads, in every - run, `pipe_copy_bytes` under 0.01 per byte downloaded, its C0, hold and spin - per MB posted beside the scout's rows. No tree that lands has that counter, - which round 3's b245592a6 added: the boot carries the scout's instrument, - the `Measurement only, never lands` commits of `origin/main..2f9e090b8` - (`wt/toyos-netring-scout`), onto stage 3's head, less the netstack stamps - on the pipe writes stage 3 deletes. + so neither drains a wake rung for the other. Within a direction no thread + holds the end across its wait, and a thread that took the doorbell while + another waits on it passes the take on through netstack. std, libc and the + mio and socket2 forks read and write the rings through `toyos::net`. Exit: + two loom models that drive the client end and netstack's ringing code as + stage 3 lands them, each red under a control that reverts its decision. In + one, a stream's reader and writer, each asleep on its own want while + netstack's thread releases room and publishes data in either order, end + with neither waiting, and the control rings both wants on one doorbell. In + the other, a blocking call and a `poll` of one direction, the poll's watch + answered by the doorbell's level at its look as the kernel's `deliver` + answers it and the blocking call leaving bytes, the receive ring's `end` or + room behind, end with the poll answered, and the control does not pass the + take on. And a T14 boot of round 3's `internet_download` on stage 3's head + reads, in every run, `pipe_copy_bytes` under 0.01 per byte downloaded, its + C0, hold and spin per MB posted beside the scout's rows. No tree that lands + has that row or that counter: the boot carries the scout's instrument, + `git diff 777c96b2b 2f9e090b8` (`wt/toyos-netring-scout` over its base), + onto stage 3's head, less the netstack stamps on the pipe writes stage 3 + deletes. - Stage 4: a listener and a datagram socket the same way, and the pipe ABI's messages, `SYS_CONNECTION_JOIN` and `toyos::net`'s pipe handles go. From d0181491bdd1e35c05a8b9a1eaefcabc9b43faa0 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 16:13:55 +0200 Subject: [PATCH 06/14] issues: the netring track's stages 1 and 2 are done Stage 1 landed as #867 and stage 2 as #878. #867 deleted issues/a-file-server-maps-a-lent-window-at-the-size-it-expects.md, so stage 1 no longer cites it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...stack-moves-every-byte-through-two-kernel-pipes.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 666f592403..0dffc5006d 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -48,13 +48,12 @@ The design and its roast are PR #863's body. - Stage 0, done: the scout boot on the T14 that splits the lock's hold into its two copies (PR #863). -- Stage 1, PR #867: the kernel's one new answer is `SYS_FSTAT`'s +- Stage 1, done (PR #867): the kernel's one new answer is `SYS_FSTAT`'s `DeviceMemory`, and a receiver judges a lent region through `fstat` before - it maps it (closes - `issues/a-file-server-maps-a-lent-window-at-the-size-it-expects.md`). -- Stage 2, PR #878: a byte ring with untrusted cursors and want words in - `toyos-transport` (`lib/blockring/transport`), host-tested and modelled in - loom. + it maps it. +- Stage 2, done (PR #878): a byte ring with untrusted cursors and want words + in `toyos-transport` (`lib/blockring/transport`), host-tested and modelled + in loom. - Stage 3: a TCP stream is one region and one connection: netstack's receive and send buffers are the region's two rings, and the connection carries the connect, then only doorbells and the hang-up. netstack rings a reader on the From c97b3fb345dd8faf6c890d977cc185269a5e8fef Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 17:29:23 +0200 Subject: [PATCH 07/14] issues: the netring track's stages 3 and 4 carry amendments C, D and E's exits The orchestrator accepted amendments C, D and E to #863's design (https://github.com/ToyOSOrg/ToyOS/pull/863#issuecomment-6127307681), as proposed in https://github.com/ToyOSOrg/ToyOS/pull/863#issuecomment-6127271821. - C: a passive child's bytes wait in [tcp]'s heap storage until its accept, which copies them into the region, as mTCP does. Stage 3 keeps that storage beside the region; stage 4's exit carries the two listeners.rs cases that pin those bytes, red when the accept drops them. - D: a registration arms the want of each direction it names and rings netstack if the direction is already ready, as epoll's ep_insert reports a ready source. Stage 3's exit carries a third loom model, red without D, red with the arm or the ring removed, and red under want-relaxed. - E: MsgType::Region goes; a connection carries the connect and its answer, then only doorbells, and a stream belongs to the process that connected or accepted it. Stage 3's exit carries a test of toyos::net's refusal of a handle it holds no end for. The track's exit now names stage 4's exit beside stage 3's. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...ves-every-byte-through-two-kernel-pipes.md | 50 +++++++++++++------ 1 file changed, 35 insertions(+), 15 deletions(-) diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 0dffc5006d..4892e4f1da 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -55,34 +55,54 @@ The design and its roast are PR #863's body. in `toyos-transport` (`lib/blockring/transport`), host-tested and modelled in loom. - Stage 3: a TCP stream is one region and one connection: netstack's receive - and send buffers are the region's two rings, and the connection carries the - connect, then only doorbells and the hang-up. netstack rings a reader on the + and send buffers are the region's two rings, beside [tcp]'s heap storage, + which stays for stage 4's passive children, and the connection carries the + connect and its answer, then only doorbells and the hang-up. A stream + belongs to the process that connected or accepted it: `toyos::net` refuses + by name a handle it holds no end for. netstack rings a reader on the connection and a writer waiting for send room on a pipe of the stream's own, so neither drains a wake rung for the other. Within a direction no thread holds the end across its wait, and a thread that took the doorbell while - another waits on it passes the take on through netstack. std, libc and the - mio and socket2 forks read and write the rings through `toyos::net`. Exit: - two loom models that drive the client end and netstack's ringing code as + another waits on it passes the take on through netstack. A registration + arms the want of each direction it names and rings netstack if that + direction is already ready, and netstack rings back. std, libc and the mio + and socket2 forks read and write the rings through `toyos::net`. Exit: + three loom models that drive the client end and netstack's ringing code as stage 3 lands them, each red under a control that reverts its decision. In one, a stream's reader and writer, each asleep on its own want while netstack's thread releases room and publishes data in either order, end with neither waiting, and the control rings both wants on one doorbell. In - the other, a blocking call and a `poll` of one direction, the poll's watch + another, a blocking call and a `poll` of one direction, the poll's watch answered by the doorbell's level at its look as the kernel's `deliver` answers it and the blocking call leaving bytes, the receive ring's `end` or room behind, end with the poll answered, and the control does not pass the - take on. And a T14 boot of round 3's `internet_download` on stage 3's head - reads, in every run, `pipe_copy_bytes` under 0.01 per byte downloaded, its - C0, hold and spin per MB posted beside the scout's rows. No tree that lands - has that row or that counter: the boot carries the scout's instrument, - `git diff 777c96b2b 2f9e090b8` (`wt/toyos-netring-scout` over its base), - onto stage 3's head, less the netstack stamps on the pipe writes stage 3 - deletes. + take on. In the third, a source registered while its direction is ready, + in both directions and with the receive ring's `end`, made ready before the + registration or racing it, gets its first event; it is red without the + registration's arm and ring, red with either of the two removed, and red + under stage 2's `want-relaxed`. A test: `toyos::net`, handed a stream + handle the process holds no end for, refuses it by name and writes nothing + on its connection. And a T14 boot of round 3's `internet_download` on stage + 3's head reads, in every run, `pipe_copy_bytes` under 0.01 per byte + downloaded, its C0, hold and spin per MB posted beside the scout's rows. No + tree that lands has that row or that counter: the boot carries the scout's + instrument, `git diff 777c96b2b 2f9e090b8` (`wt/toyos-netring-scout` over + its base), onto stage 3's head, less the netstack stamps on the pipe writes + stage 3 deletes. - Stage 4: a listener and a datagram socket the same way, and the pipe ABI's - messages, `SYS_CONNECTION_JOIN` and `toyos::net`'s pipe handles go. + messages, `SYS_CONNECTION_JOIN` and `toyos::net`'s pipe handles go. A + passive child's bytes wait in [tcp]'s heap storage, at most + `limits::RECEIVE_BUFFER_INITIAL` a child and `limits::LISTEN_READY` + children a listener, and its accept copies them into the region before it + answers. Exit: + `system/netstack/node/tests/listeners.rs`'s + `a_listener_answers_a_syn_and_wakes_its_owner_when_the_handshake_ends` and + `an_accept_answers_the_option_of_a_stream_its_own_pass_let_go` pass over + the region, and each is red when what a child held is dropped at its + accept. Owner: the orchestrator, who stages it. -Exit: stage 3's exit, and the pipe ABI gone with stage 4: +Exit: stage 3's and stage 4's exits, and the pipe ABI gone with stage 4: `rg -n 'Piped|DATA_HANDLES|NOTIFY_HANDLES|connection_join|CONNECTION_JOIN' sdk system kernel` and `rg -n 'pipe_pair' sdk/toyos/src/net.rs` are empty. From 68b26142a6448134b60cebbf8062c1400d097546 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 20:24:18 +0200 Subject: [PATCH 08/14] issues: stage 4's exit takes a child's text past a hole and its FIN, a fatal end readies the send direction, and a stream cannot move between processes Round 4's review of #863 (issuecomment-6129829193) and the orchestrator's rulings on it (issuecomment-6129844256): - Stage 4's exit drops `an_accept_answers_the_option_of_a_stream_its_own_pass_let_go`, which reds only through `ToClient`'s `WriteRefusal::Broken`, both on stage 4's deletion list. In its place goes amendment C's own case: a child holds, before its accept, text in order, text past a hole and its FIN behind that; the hole is filled after the accept, and the region reads all of it in order, then its `end`. It is red under each of three controls dropping at the accept one thing the child held. Run against today's node tests, each control reds the case: the in-order bytes and the out-of-order range as zeros where text was, the FIN as an end that never comes. - A fatal `end`, any but a FIN, makes the send direction ready to every look, as Linux v6.12's `tcp_poll` answers a reset socket `EPOLLOUT` once `tcp_done` has set `SHUTDOWN_MASK`. Without it a writer or a `WRITABLE` registration that arms after a reset while the send ring is full looks only at the head and sleeps on a bell nobody rings again. Stage 3's third model carries the case and a control whose looks read only the head. - Stage 3 says std gains `FromRawFd`, `IntoRawFd` and the `OwnedFd` conversions for `TcpStream`, taking only the process's own ends, and that `OwnedFd`'s drop lets a stream's handle go through `toyos::net`, so no entry outlives its handle's number. - `issues/a-stream-cannot-move-between-processes.md` files the gap amendment E leaves, with its owner and E's exit. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- .../a-stream-cannot-move-between-processes.md | 30 ++++++++ ...ves-every-byte-through-two-kernel-pipes.md | 75 +++++++++++-------- 2 files changed, 72 insertions(+), 33 deletions(-) create mode 100644 issues/a-stream-cannot-move-between-processes.md diff --git a/issues/a-stream-cannot-move-between-processes.md b/issues/a-stream-cannot-move-between-processes.md new file mode 100644 index 0000000000..daa1ed33a2 --- /dev/null +++ b/issues/a-stream-cannot-move-between-processes.md @@ -0,0 +1,30 @@ +--- +status: open +kind: defect +opened: 2026-10-12 +--- + +# A stream cannot move between processes + +A program that hands a TCP stream to another process, as socket activation or +an inetd-style child does, cannot do so on ToyOS, though root `CLAUDE.md` says +existing Rust just works. A process holds a stream only by connecting or +accepting it. At ab655a115 nothing in the tree takes a stream handle another +process made: std on ToyOS has no socket `FromRawFd` or `From` (the +`rust/` fork's `library/std/src/os/fd/mod.rs` compiles `net` out for `toyos`), +libc's and socket2's sockets are keys in their own process's tables +(`SOCKET_FD_BASE`, `sdk/libc/src/socket.rs`; socket2's `NEXT_FD`), and the mio +fork's ToyOS stream has no `from_std`. + +The session design keeps it so +(`issues/netstack-moves-every-byte-through-two-kernel-pipes.md`, stage 3; PR +#863, amendment E): after its answer a stream's connection carries only +doorbells, so a process handed the handle has no channel on which to ask +netstack for the stream's region, and `toyos::net` refuses by name a handle the +process holds no end for, std's `FromRawFd` included once stage 3 adds it. + +Owner: the orchestrator, who stages +`issues/netstack-moves-every-byte-through-two-kernel-pipes.md`. + +Exit: a stream moved by its handle reads and writes in the receiving process, +and the process that moved it holds no end. diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 4892e4f1da..10d042b1bd 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -59,31 +59,38 @@ The design and its roast are PR #863's body. which stays for stage 4's passive children, and the connection carries the connect and its answer, then only doorbells and the hang-up. A stream belongs to the process that connected or accepted it: `toyos::net` refuses - by name a handle it holds no end for. netstack rings a reader on the - connection and a writer waiting for send room on a pipe of the stream's own, - so neither drains a wake rung for the other. Within a direction no thread - holds the end across its wait, and a thread that took the doorbell while - another waits on it passes the take on through netstack. A registration - arms the want of each direction it names and rings netstack if that - direction is already ready, and netstack rings back. std, libc and the mio - and socket2 forks read and write the rings through `toyos::net`. Exit: - three loom models that drive the client end and netstack's ringing code as - stage 3 lands them, each red under a control that reverts its decision. In - one, a stream's reader and writer, each asleep on its own want while - netstack's thread releases room and publishes data in either order, end - with neither waiting, and the control rings both wants on one doorbell. In - another, a blocking call and a `poll` of one direction, the poll's watch - answered by the doorbell's level at its look as the kernel's `deliver` - answers it and the blocking call leaving bytes, the receive ring's `end` or - room behind, end with the poll answered, and the control does not pass the - take on. In the third, a source registered while its direction is ready, - in both directions and with the receive ring's `end`, made ready before the - registration or racing it, gets its first event; it is red without the - registration's arm and ring, red with either of the two removed, and red - under stage 2's `want-relaxed`. A test: `toyos::net`, handed a stream - handle the process holds no end for, refuses it by name and writes nothing - on its connection. And a T14 boot of round 3's `internet_download` on stage - 3's head reads, in every run, `pipe_copy_bytes` under 0.01 per byte + by name a handle it holds no end for + (`issues/a-stream-cannot-move-between-processes.md`). netstack rings a + reader on the connection and a writer waiting for send room on a pipe of the + stream's own, so neither drains a wake rung for the other. Within a + direction no thread holds the end across its wait, and a thread that took + the doorbell while another waits on it passes the take on through netstack. + A registration arms the want of each direction it names and rings netstack + if that direction is already ready, and netstack rings back. A fatal `end`, + any but a FIN, makes the send direction ready to every look, as Linux's + `tcp_poll` answers a reset socket writable. std, libc and the mio and + socket2 forks read and write the rings through `toyos::net`, and std gains + `FromRawFd`, `IntoRawFd` and the `OwnedFd` conversions for `TcpStream`, + which take only the process's own ends, with `OwnedFd`'s drop letting a + stream's handle go through `toyos::net`. Exit: three loom models that drive + the client end and netstack's ringing code as stage 3 lands them, each red + under a control that reverts its decision. In one, a stream's reader and + writer, each asleep on its own want while netstack's thread releases room + and publishes data in either order, end with neither waiting, and the + control rings both wants on one doorbell. In another, a blocking call and a + `poll` of one direction, the poll's watch answered by the doorbell's level + at its look as the kernel's `deliver` answers it and the blocking call + leaving bytes, the receive ring's `end` or room behind, end with the poll + answered, and the control does not pass the take on. In the third, a source + registered while its direction is ready, in both directions, with the + receive ring's `end`, and with a fatal `end` while the send ring is full, + made ready before the registration or racing it, gets its first event; it is + red without the registration's arm and ring, red with either of the two + removed, red under stage 2's `want-relaxed`, and red with the fatal `end` + where a look reads only the send ring's head. A test: `toyos::net`, handed a + stream handle the process holds no end for, refuses it by name and writes + nothing on its connection. And a T14 boot of round 3's `internet_download` + on stage 3's head reads, in every run, `pipe_copy_bytes` under 0.01 per byte downloaded, its C0, hold and spin per MB posted beside the scout's rows. No tree that lands has that row or that counter: the boot carries the scout's instrument, `git diff 777c96b2b 2f9e090b8` (`wt/toyos-netring-scout` over @@ -92,14 +99,16 @@ The design and its roast are PR #863's body. - Stage 4: a listener and a datagram socket the same way, and the pipe ABI's messages, `SYS_CONNECTION_JOIN` and `toyos::net`'s pipe handles go. A passive child's bytes wait in [tcp]'s heap storage, at most - `limits::RECEIVE_BUFFER_INITIAL` a child and `limits::LISTEN_READY` - children a listener, and its accept copies them into the region before it - answers. Exit: - `system/netstack/node/tests/listeners.rs`'s - `a_listener_answers_a_syn_and_wakes_its_owner_when_the_handshake_ends` and - `an_accept_answers_the_option_of_a_stream_its_own_pass_let_go` pass over - the region, and each is red when what a child held is dropped at its - accept. + `limits::RECEIVE_BUFFER_INITIAL` a child and `limits::LISTEN_READY` children + a listener, and its accept copies them into the region before it answers. + Exit: over the region, `system/netstack/node/tests/listeners.rs`'s + `a_listener_answers_a_syn_and_wakes_its_owner_when_the_handshake_ends` + passes, and so does a case in which a child holds, before its accept, text + in order, text past a hole and its FIN behind that, the hole is filled after + the accept, and the region reads all of it in order, then its `end`. That + case is red under each of three controls that drop at the accept one thing + the child held: its in-order bytes, which also reds the first, its + out-of-order range, its FIN. Owner: the orchestrator, who stages it. From 04ff95bc785db6ef28230d4a1397c3dcbfb5a3e0 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 20:44:42 +0200 Subject: [PATCH 09/14] issues: a write after a fatal end copies nothing, an accept makes its region only for a child, and netstack judges each datagram record The orchestrator's rulings (issuecomment-6130141310) on the second triage of an external review of #863 (issuecomment-6130134066), each following Linux: - Stage 3: a `write` reads the receive ring's `end` before it copies, and a fatal one is its error with nothing copied, as v6.12's `tcp_sendmsg_locked` checks `sk_err` first. The record as it stood copied whenever the send ring had room, so after a reset every write answered `Ok` until 4 MiB less 4 KiB were queued for nobody. Stage 3's exit gains a test of it with its control. - Stage 4: an accept takes a wake from the listener's connection before it makes a region, so a nonblocking accept with nothing ready answers `WouldBlock` having made nothing, as `inet_csk_accept` answers `EAGAIN` from an empty queue; the answer carries the peer's address and the `nodelay` the child inherited from its listener. - Stage 4: netstack copies each datagram record's `len`, `port` and `addr` out of the region once and judges the copy, where today `udp_send_to` refuses a `len` naming bytes never written; its exit gains a datagram case with a control for each bound on `len`. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...ves-every-byte-through-two-kernel-pipes.md | 76 +++++++++++-------- 1 file changed, 45 insertions(+), 31 deletions(-) diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 10d042b1bd..471dc1acfd 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -68,47 +68,61 @@ The design and its roast are PR #863's body. A registration arms the want of each direction it names and rings netstack if that direction is already ready, and netstack rings back. A fatal `end`, any but a FIN, makes the send direction ready to every look, as Linux's - `tcp_poll` answers a reset socket writable. std, libc and the mio and - socket2 forks read and write the rings through `toyos::net`, and std gains - `FromRawFd`, `IntoRawFd` and the `OwnedFd` conversions for `TcpStream`, - which take only the process's own ends, with `OwnedFd`'s drop letting a - stream's handle go through `toyos::net`. Exit: three loom models that drive - the client end and netstack's ringing code as stage 3 lands them, each red - under a control that reverts its decision. In one, a stream's reader and - writer, each asleep on its own want while netstack's thread releases room - and publishes data in either order, end with neither waiting, and the - control rings both wants on one doorbell. In another, a blocking call and a - `poll` of one direction, the poll's watch answered by the doorbell's level - at its look as the kernel's `deliver` answers it and the blocking call - leaving bytes, the receive ring's `end` or room behind, end with the poll - answered, and the control does not pass the take on. In the third, a source - registered while its direction is ready, in both directions, with the - receive ring's `end`, and with a fatal `end` while the send ring is full, - made ready before the registration or racing it, gets its first event; it is - red without the registration's arm and ring, red with either of the two - removed, red under stage 2's `want-relaxed`, and red with the fatal `end` - where a look reads only the send ring's head. A test: `toyos::net`, handed a - stream handle the process holds no end for, refuses it by name and writes - nothing on its connection. And a T14 boot of round 3's `internet_download` - on stage 3's head reads, in every run, `pipe_copy_bytes` under 0.01 per byte - downloaded, its C0, hold and spin per MB posted beside the scout's rows. No - tree that lands has that row or that counter: the boot carries the scout's - instrument, `git diff 777c96b2b 2f9e090b8` (`wt/toyos-netring-scout` over - its base), onto stage 3's head, less the netstack stamps on the pipe writes - stage 3 deletes. + `tcp_poll` answers a reset socket writable, and a `write` reads it before it + copies, answering its error and copying nothing, as Linux checks `sk_err` + first. std, libc and the mio and socket2 forks read and write the rings + through `toyos::net`, and std gains `FromRawFd`, `IntoRawFd` and the + `OwnedFd` conversions for `TcpStream`, which take only the process's own + ends, with `OwnedFd`'s drop letting a stream's handle go through + `toyos::net`. Exit: three loom models that drive the client end and + netstack's ringing code as stage 3 lands them, each red under a control that + reverts its decision. In one, a stream's reader and writer, each asleep on + its own want while netstack's thread releases room and publishes data in + either order, end with neither waiting, and the control rings both wants on + one doorbell. In another, a blocking call and a `poll` of one direction, the + poll's watch answered by the doorbell's level at its look as the kernel's + `deliver` answers it and the blocking call leaving bytes, the receive ring's + `end` or room behind, end with the poll answered, and the control does not + pass the take on. In the third, a source registered while its direction is + ready, in both directions, with the receive ring's `end`, and with a fatal + `end` while the send ring is full, made ready before the registration or + racing it, gets its first event; it is red without the registration's arm + and ring, red with either of the two removed, red under stage 2's + `want-relaxed`, and red with the fatal `end` where a look reads only the + send ring's head. A test: `toyos::net`, handed a stream handle the process + holds no end for, refuses it by name and writes nothing on its connection. A + test: a `write` with room after a reset answers `ConnectionReset` and copies + nothing, red when `write` copies before it reads the `end`. And a T14 boot + of round 3's `internet_download` on stage 3's head reads, in every run, + `pipe_copy_bytes` under 0.01 per byte downloaded, its C0, hold and spin per + MB posted beside the scout's rows. No tree that lands has that row or that + counter: the boot carries the scout's instrument, `git diff 777c96b2b + 2f9e090b8` (`wt/toyos-netring-scout` over its base), onto stage 3's head, + less the netstack stamps on the pipe writes stage 3 deletes. - Stage 4: a listener and a datagram socket the same way, and the pipe ABI's messages, `SYS_CONNECTION_JOIN` and `toyos::net`'s pipe handles go. A passive child's bytes wait in [tcp]'s heap storage, at most `limits::RECEIVE_BUFFER_INITIAL` a child and `limits::LISTEN_READY` children - a listener, and its accept copies them into the region before it answers. - Exit: over the region, `system/netstack/node/tests/listeners.rs`'s + a listener, and its accept copies them into the region before it answers. An + accept takes one of the wakes netstack writes on the listener's connection + for each child that waits before it makes a region, so a nonblocking accept + with none answers `WouldBlock` having made nothing, and the accept's answer + carries the peer's address and the `nodelay` the child inherited from its + listener. netstack copies each datagram record's `len`, `port` and `addr` + out of the region once and judges the copy: a `len` past the bytes the + record holds ends that socket alone, and one past `MAX_PAYLOAD`, port 0 or + an address [udp] refuses is refused by its counter. Exit: over the region, + `system/netstack/node/tests/listeners.rs`'s `a_listener_answers_a_syn_and_wakes_its_owner_when_the_handshake_ends` passes, and so does a case in which a child holds, before its accept, text in order, text past a hole and its FIN behind that, the hole is filled after the accept, and the region reads all of it in order, then its `end`. That case is red under each of three controls that drop at the accept one thing the child held: its in-order bytes, which also reds the first, its - out-of-order range, its FIN. + out-of-order range, its FIN. And a datagram record whose `len` is past the + bytes it holds, and one whose `len` is past `MAX_PAYLOAD`, are each refused + while netstack serves on, each red when netstack takes that `len` as + written. Owner: the orchestrator, who stages it. From 0831ca65f89f33dae5b08591ed4c49a01550124f Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 21:26:50 +0200 Subject: [PATCH 10/14] issues: a stream's handle in a File or a child's stdio is refused loudly, the E test gains its boundary, and the track cites RFC 9293 and POSIX where it cited Linux MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 5's review (PR #863, 6130914125) and the orchestrator's ruling on E (6130929883): - The stream-move issue states E's cost as the regression it is. Today a stream's handle is its joined data pipes, and `File::from_raw_fd` and `Stdio::from(File)` read and write it with the same `syscall::read` and `write` as `TcpStream`; over the region that handle is the doorbell connection. Stage 3 makes `File::from_raw_fd` of a stream's handle panic with `toyos::net`'s refusal and a spawn that would put one in a child's stdio slot answer it, and its exit gains a test that reds when either takes the handle silently. The three claims the review quoted as false are gone. - Stage 3's E test gains its boundary (BLOCKER 3): a stream's own handle given out by `into_raw_fd` or `From for OwnedFd`, or made by `try_clone_to_owned`, and taken back, shares the one end, red when giving it out lets the entry go. The track names the `rust/` fork's ToyOS arms of `OwnedFd`'s drop and `BorrowedFd::try_clone_to_owned`, which stage 3 routes through `toyos::net` (NOTE 1, NOTE 3). - Stage 4 judges a datagram record's `len` only against the bytes the record holds, and its `MAX_PAYLOAD` case goes (BLOCKER 2): [udp]'s `submit` already refuses that payload by `Counter::ExceedsMtu`, so the case could not go red under its control. The owner's direction, relayed by the orchestrator this round: ToyOS does not follow Linux and references it nowhere in the tree. The track's two Linux citations become POSIX's `select` ("would not block, whether or not the function would transfer data successfully") for a fatal end making the send direction ready, and RFC 9293 §3.10.7.4, §3.10.8 and §3.10.2 for a write after a fatal end copying nothing. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- .../a-stream-cannot-move-between-processes.md | 43 ++++--- ...ves-every-byte-through-two-kernel-pipes.md | 105 ++++++++++-------- 2 files changed, 89 insertions(+), 59 deletions(-) diff --git a/issues/a-stream-cannot-move-between-processes.md b/issues/a-stream-cannot-move-between-processes.md index daa1ed33a2..065d207c2e 100644 --- a/issues/a-stream-cannot-move-between-processes.md +++ b/issues/a-stream-cannot-move-between-processes.md @@ -6,22 +6,37 @@ opened: 2026-10-12 # A stream cannot move between processes -A program that hands a TCP stream to another process, as socket activation or -an inetd-style child does, cannot do so on ToyOS, though root `CLAUDE.md` says -existing Rust just works. A process holds a stream only by connecting or -accepting it. At ab655a115 nothing in the tree takes a stream handle another -process made: std on ToyOS has no socket `FromRawFd` or `From` (the -`rust/` fork's `library/std/src/os/fd/mod.rs` compiles `net` out for `toyos`), -libc's and socket2's sockets are keys in their own process's tables -(`SOCKET_FD_BASE`, `sdk/libc/src/socket.rs`; socket2's `NEXT_FD`), and the mio -fork's ToyOS stream has no `from_std`. +Today a TCP stream's handle reads and writes the stream in any process that +holds it. The handle is the stream's two data pipes joined into one +connection (`make_socket_fd`, `sdk/std/sys/net/connection.rs`), which +`TcpStream` reads and writes with `syscall::read` and `write`. +`File::from_raw_fd` takes any handle (`sdk/std/os/io.rs`) and reads and writes +it with the same calls (`sdk/std/sys/fs.rs`), and `Stdio::from(File)` hands +it to a child, whose slot the kernel fills with a duplicate +(`sdk/std/sys/process.rs`; `build_child_handles`, +`kernel/src/loader/start.rs`). So an inetd-style child serves the stream on +its stdin and stdout for as long as the process that connected it holds its +`TcpStream`. -The session design keeps it so +The session design takes that away (`issues/netstack-moves-every-byte-through-two-kernel-pipes.md`, stage 3; PR -#863, amendment E): after its answer a stream's connection carries only -doorbells, so a process handed the handle has no channel on which to ask -netstack for the stream's region, and `toyos::net` refuses by name a handle the -process holds no end for, std's `FromRawFd` included once stage 3 adds it. +#863, amendment E). After its answer, a stream's connection carries only +doorbells, and the region that holds the stream's bytes is mapped only by +the process that connected or accepted it. Read through a `File` or a +child's stdio, the handle would wait for ever or read doorbells as data, and +netstack would drain what the child writes as doorbells, with no error on +either side. Stage 3 makes std refuse both loudly instead. +`File::from_raw_fd` of a stream's handle panics with `toyos::net`'s refusal. +A spawn that would put a stream's handle in a child's stdio slot answers that +refusal and starts no child. And `toyos::net` refuses by name a handle the +process holds no end for, std's `TcpStream::from_raw_fd` included. A program +that hands a stream to another process through a `File` or a child's stdio, +as an inetd-style server does, then fails loudly on ToyOS where today it +works, and one that takes a moved stream back as a `TcpStream`, as socket +activation does and as no ToyOS std builds today, is refused, though root +`CLAUDE.md` says existing Rust just works. ToyOS's own +`CommandExt::inherit_handle` past the stdio slots and `CommandExt::endow` +still hand a child a stream's handle with no refusal. Owner: the orchestrator, who stages `issues/netstack-moves-every-byte-through-two-kernel-pipes.md`. diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 471dc1acfd..19f0b0b1c0 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -59,46 +59,61 @@ The design and its roast are PR #863's body. which stays for stage 4's passive children, and the connection carries the connect and its answer, then only doorbells and the hang-up. A stream belongs to the process that connected or accepted it: `toyos::net` refuses - by name a handle it holds no end for - (`issues/a-stream-cannot-move-between-processes.md`). netstack rings a - reader on the connection and a writer waiting for send room on a pipe of the - stream's own, so neither drains a wake rung for the other. Within a - direction no thread holds the end across its wait, and a thread that took - the doorbell while another waits on it passes the take on through netstack. - A registration arms the want of each direction it names and rings netstack - if that direction is already ready, and netstack rings back. A fatal `end`, - any but a FIN, makes the send direction ready to every look, as Linux's - `tcp_poll` answers a reset socket writable, and a `write` reads it before it - copies, answering its error and copying nothing, as Linux checks `sk_err` - first. std, libc and the mio and socket2 forks read and write the rings - through `toyos::net`, and std gains `FromRawFd`, `IntoRawFd` and the - `OwnedFd` conversions for `TcpStream`, which take only the process's own - ends, with `OwnedFd`'s drop letting a stream's handle go through - `toyos::net`. Exit: three loom models that drive the client end and - netstack's ringing code as stage 3 lands them, each red under a control that - reverts its decision. In one, a stream's reader and writer, each asleep on - its own want while netstack's thread releases room and publishes data in - either order, end with neither waiting, and the control rings both wants on - one doorbell. In another, a blocking call and a `poll` of one direction, the - poll's watch answered by the doorbell's level at its look as the kernel's - `deliver` answers it and the blocking call leaving bytes, the receive ring's - `end` or room behind, end with the poll answered, and the control does not - pass the take on. In the third, a source registered while its direction is - ready, in both directions, with the receive ring's `end`, and with a fatal - `end` while the send ring is full, made ready before the registration or - racing it, gets its first event; it is red without the registration's arm - and ring, red with either of the two removed, red under stage 2's - `want-relaxed`, and red with the fatal `end` where a look reads only the - send ring's head. A test: `toyos::net`, handed a stream handle the process - holds no end for, refuses it by name and writes nothing on its connection. A - test: a `write` with room after a reset answers `ConnectionReset` and copies - nothing, red when `write` copies before it reads the `end`. And a T14 boot - of round 3's `internet_download` on stage 3's head reads, in every run, - `pipe_copy_bytes` under 0.01 per byte downloaded, its C0, hold and spin per - MB posted beside the scout's rows. No tree that lands has that row or that - counter: the boot carries the scout's instrument, `git diff 777c96b2b - 2f9e090b8` (`wt/toyos-netring-scout` over its base), onto stage 3's head, - less the netstack stamps on the pipe writes stage 3 deletes. + by name a handle it holds no end for. netstack rings a reader on the + connection and a writer waiting for send room on a pipe of the stream's + own, so neither drains a wake rung for the other. Within a direction no + thread holds the end across its wait, and a thread that took the doorbell + while another waits on it passes the take on through netstack. A + registration arms the want of each direction it names and rings netstack if + that direction is already ready, and netstack rings back. A fatal `end`, + any but a FIN, makes the send direction ready to every look, as POSIX's + `select` counts a descriptor ready for writing when a write "would not + block, whether or not the function would transfer data successfully", and a + `write` reads it before it copies, answering its error and copying nothing, + as RFC 9293 flushes every segment queue at a reset or a user timeout + (§3.10.7.4, §3.10.8) and answers a SEND that finds no connection an error + (§3.10.2). std, libc and the mio and socket2 forks read and write the rings + through `toyos::net`. std gains `FromRawFd`, `IntoRawFd` and the `OwnedFd` + conversions for `TcpStream`, which take only the process's own ends, and + the `rust/` fork's ToyOS arms of `OwnedFd`'s drop and + `BorrowedFd::try_clone_to_owned` let go of and enter a stream's handle + through `toyos::net`. Today a stream's handle reads + and writes the stream through a `File` or a child's stdio; over the region + it would read doorbells, so `File::from_raw_fd` of a stream's handle panics + with `toyos::net`'s refusal and a spawn that would put one in a child's + stdio slot answers it and starts no child: a regression, + `issues/a-stream-cannot-move-between-processes.md`. Exit: three loom models + that drive the client end and netstack's ringing code as stage 3 lands + them, each red under a control that reverts its decision. In one, a + stream's reader and writer, each asleep on its own want while netstack's + thread releases room and publishes data in either order, end with neither + waiting, and the control rings both wants on one doorbell. In another, a + blocking call and a `poll` of one direction, the poll's watch answered by + the doorbell's level at its look as the kernel's `deliver` answers it and + the blocking call leaving bytes, the receive ring's `end` or room behind, + end with the poll answered, and the control does not pass the take on. In + the third, a source registered while its direction is ready, in both + directions, with the receive ring's `end`, and with a fatal `end` while the + send ring is full, made ready before the registration or racing it, gets + its first event; it is red without the registration's arm and ring, red + with either of the two removed, red under stage 2's `want-relaxed`, and red + with the fatal `end` where a look reads only the send ring's head. A test: + `toyos::net`, handed a stream handle the process holds no end for, refuses + it by name and writes nothing on its connection, and a stream's own handle, + given out by `into_raw_fd` or `From for OwnedFd` or made by + `try_clone_to_owned`, and taken back, shares the one end and its cursors, + red when giving the handle out lets its entry go. A test: + `File::from_raw_fd` of a stream's handle panics with that refusal, and a + spawn whose stdio slot holds one answers it and starts no child, each red + when it takes the handle silently. A test: a `write` with room after a + reset answers `ConnectionReset` and copies nothing, red when `write` copies + before it reads the `end`. And a T14 boot of round 3's `internet_download` + on stage 3's head reads, in every run, `pipe_copy_bytes` under 0.01 per + byte downloaded, its C0, hold and spin per MB posted beside the scout's + rows. No tree that lands has that row or that counter: the boot carries the + scout's instrument, `git diff 777c96b2b 2f9e090b8` + (`wt/toyos-netring-scout` over its base), onto stage 3's head, less the + netstack stamps on the pipe writes stage 3 deletes. - Stage 4: a listener and a datagram socket the same way, and the pipe ABI's messages, `SYS_CONNECTION_JOIN` and `toyos::net`'s pipe handles go. A passive child's bytes wait in [tcp]'s heap storage, at most @@ -110,8 +125,9 @@ The design and its roast are PR #863's body. carries the peer's address and the `nodelay` the child inherited from its listener. netstack copies each datagram record's `len`, `port` and `addr` out of the region once and judges the copy: a `len` past the bytes the - record holds ends that socket alone, and one past `MAX_PAYLOAD`, port 0 or - an address [udp] refuses is refused by its counter. Exit: over the region, + record holds ends that socket alone, and [udp] refuses by its counter, as + it does today, a payload past `MAX_PAYLOAD`, port 0 or an address it + cannot send to. Exit: over the region, `system/netstack/node/tests/listeners.rs`'s `a_listener_answers_a_syn_and_wakes_its_owner_when_the_handshake_ends` passes, and so does a case in which a child holds, before its accept, text @@ -120,9 +136,8 @@ The design and its roast are PR #863's body. case is red under each of three controls that drop at the accept one thing the child held: its in-order bytes, which also reds the first, its out-of-order range, its FIN. And a datagram record whose `len` is past the - bytes it holds, and one whose `len` is past `MAX_PAYLOAD`, are each refused - while netstack serves on, each red when netstack takes that `len` as - written. + bytes it holds is refused while netstack serves on, red when netstack takes + that `len` as written. Owner: the orchestrator, who stages it. From 9b860ac69e2b4a6b02fd35840652145dadbee61e Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 03:11:43 +0200 Subject: [PATCH 11/14] issues: every path that moves a handle refuses a stream's, a refused datagram is never Ok, and the track cites no implementation Round 6's review of #863 and the orchestrator's rulings on it (issuecomment-6135709402): - Stage 3: a spawn's stdio slots, CommandExt::inherit_handle into any slot, CommandExt::endow, Command::provide and toyos::ipc's send_handles each ask toyos::net before anything moves and answer its refusal, so no refused move leaves an entry that outlives its number. The bare dup, SYS_HANDLE_SEND and SYS_SPAWN stay unchecked. The exit gains a case per path; endow's and provide's red when the handle moves and its entry stays. - Stage 4: toyos::net's send_to refuses what a sender can judge (a payload past a bound toyos-abi declares and [udp] reads, port 0, an unspecified address) and writes no record; what only netstack can judge is the socket's pending error on its next call (POSIX XSH 2.10.10). The exit gains a case per refusal, red when it answers Ok. - The stream-move issue names every path and drops "unrefused". Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- .../a-stream-cannot-move-between-processes.md | 22 +++++----- ...ves-every-byte-through-two-kernel-pipes.md | 44 +++++++++++++++---- 2 files changed, 47 insertions(+), 19 deletions(-) diff --git a/issues/a-stream-cannot-move-between-processes.md b/issues/a-stream-cannot-move-between-processes.md index 065d207c2e..6160580c7b 100644 --- a/issues/a-stream-cannot-move-between-processes.md +++ b/issues/a-stream-cannot-move-between-processes.md @@ -27,16 +27,18 @@ child's stdio, the handle would wait for ever or read doorbells as data, and netstack would drain what the child writes as doorbells, with no error on either side. Stage 3 makes std refuse both loudly instead. `File::from_raw_fd` of a stream's handle panics with `toyos::net`'s refusal. -A spawn that would put a stream's handle in a child's stdio slot answers that -refusal and starts no child. And `toyos::net` refuses by name a handle the -process holds no end for, std's `TcpStream::from_raw_fd` included. A program -that hands a stream to another process through a `File` or a child's stdio, -as an inetd-style server does, then fails loudly on ToyOS where today it -works, and one that takes a moved stream back as a `TcpStream`, as socket -activation does and as no ToyOS std builds today, is refused, though root -`CLAUDE.md` says existing Rust just works. ToyOS's own -`CommandExt::inherit_handle` past the stdio slots and `CommandExt::endow` -still hand a child a stream's handle with no refusal. +Every path that moves a handle into another process answers that refusal +before anything moves: a spawn's stdio slots, `CommandExt::inherit_handle` +into any slot, `CommandExt::endow` and `Command::provide`, each starting no +child, and `toyos::ipc`'s `send_handles`, which sends nothing. And +`toyos::net` refuses by name a handle the process holds no end for, std's +`TcpStream::from_raw_fd` included. A program that hands a stream to another +process by any of these, as an inetd-style server does through its child's +stdio, then fails loudly on ToyOS where today it works, and one that takes a +moved stream back as a `TcpStream`, as socket activation does and as no +ToyOS std builds today, is refused, though root `CLAUDE.md` says existing +Rust just works. The bare `dup`, `SYS_HANDLE_SEND` and `SYS_SPAWN` syscalls +are not checked. Owner: the orchestrator, who stages `issues/netstack-moves-every-byte-through-two-kernel-pipes.md`. diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 19f0b0b1c0..d08b789898 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -80,8 +80,15 @@ The design and its roast are PR #863's body. through `toyos::net`. Today a stream's handle reads and writes the stream through a `File` or a child's stdio; over the region it would read doorbells, so `File::from_raw_fd` of a stream's handle panics - with `toyos::net`'s refusal and a spawn that would put one in a child's - stdio slot answers it and starts no child: a regression, + with `toyos::net`'s refusal, and every path that moves a handle into + another process asks `toyos::net` of each handle before anything moves and + answers its refusal: a spawn's stdio slots, `CommandExt::inherit_handle` + into any slot, `CommandExt::endow` and `Command::provide`, each with no + child started and the handle still the caller's, and `toyos::ipc`'s + `send_handles`, which sends nothing and lets its batch go as every refused + move does, a stream's handle through `toyos::net`. So no refused move + leaves an entry that outlives its handle's number. The bare `dup`, + `SYS_HANDLE_SEND` and `SYS_SPAWN` syscalls stay unchecked. A regression, `issues/a-stream-cannot-move-between-processes.md`. Exit: three loom models that drive the client end and netstack's ringing code as stage 3 lands them, each red under a control that reverts its decision. In one, a @@ -103,9 +110,16 @@ The design and its roast are PR #863's body. given out by `into_raw_fd` or `From for OwnedFd` or made by `try_clone_to_owned`, and taken back, shares the one end and its cursors, red when giving the handle out lets its entry go. A test: - `File::from_raw_fd` of a stream's handle panics with that refusal, and a - spawn whose stdio slot holds one answers it and starts no child, each red - when it takes the handle silently. A test: a `write` with room after a + `File::from_raw_fd` of a stream's handle panics with that refusal, red when + it takes the handle silently. A test per moving path, each handed a + stream's handle: a spawn's stdio slot, `inherit_handle` into a slot past + the stdio slots, `endow` and `provide` each answer the refusal and start no + child, and the stream then reads and writes through that handle; each is + red when its path takes the handle silently, and `endow`'s and `provide`'s + are red when the handle moves and its entry stays. `send_handles` answers + the refusal and sends nothing, and the stream's entry goes with the + handle; red when it sends, and red when the refused batch closes the handle + past `toyos::net`. A test: a `write` with room after a reset answers `ConnectionReset` and copies nothing, red when `write` copies before it reads the `end`. And a T14 boot of round 3's `internet_download` on stage 3's head reads, in every run, `pipe_copy_bytes` under 0.01 per @@ -125,9 +139,18 @@ The design and its roast are PR #863's body. carries the peer's address and the `nodelay` the child inherited from its listener. netstack copies each datagram record's `len`, `port` and `addr` out of the region once and judges the copy: a `len` past the bytes the - record holds ends that socket alone, and [udp] refuses by its counter, as - it does today, a payload past `MAX_PAYLOAD`, port 0 or an address it - cannot send to. Exit: over the region, + record holds ends that socket alone. `toyos::net`'s `send_to` refuses + what a sender can judge and writes no record: a payload past the bound + that `toyos-abi`'s `net` module declares and [udp] reads in place of its + own `limits::MAX_PAYLOAD`, as POSIX's `sendto` fails `[EMSGSIZE]`; port 0; + an unspecified address. [udp] still refuses each by its counter at the + trust boundary. What only netstack can judge, a connected socket's other + destination, a route, broadcast permission, a source address and a pending + ICMP error, netstack stores in the region's `pending` word and rings, and + the socket's next call answers it and clears it, as POSIX's pending error + is "returned in response to a subsequent send(), recv(), or getsockopt() + operation" (XSH 2.10.10). netstack takes a record only while [udp]'s send + queue has room. Exit: over the region, `system/netstack/node/tests/listeners.rs`'s `a_listener_answers_a_syn_and_wakes_its_owner_when_the_handshake_ends` passes, and so does a case in which a child holds, before its accept, text @@ -137,7 +160,10 @@ The design and its roast are PR #863's body. the child held: its in-order bytes, which also reds the first, its out-of-order range, its FIN. And a datagram record whose `len` is past the bytes it holds is refused while netstack serves on, red when netstack takes - that `len` as written. + that `len` as written. A case per refusal: an oversized payload, port 0 + and an unspecified address each answer their error at `send_to` with no + record written, and each refusal only netstack judges answers on the + socket's next call; each is red when that call answers `Ok`. Owner: the orchestrator, who stages it. From 53c2d8429cf933b3dc5e02da84656b817a9793d6 Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 03:50:35 +0200 Subject: [PATCH 12/14] issues: send_handles is refused by its type, a datagram's pending error wakes every sleeper, and a sender waits at a full queue Round 7's review of #863 (SEND BACK) and the orchestrator's ruling on send_handles. - send_handles takes OwnedHandle, which safe code makes only from a Pipe: a stream's handle reaches it only by breaking OwnedHandle::from_raw's contract, which stage 3 extends to name it. The check, its exit case and its lookup go from the track and the stream-move issue. - netstack stores a datagram socket's pending error, then exchanges both client wants and rings each that was set; a look after the arm reads pending beside its ring. Stage 4's exit gains a loom model that reds when the look reads pending before the arm, when the wants go before the store, and when only rx_data is rung. - A full [udp] queue holds the send ring; a send_to waits for room on the socket's own bell, as POSIX's sendto blocks, and stage 4's exit gains a model that reds at a refusal at the full queue and at room rung on the connection. - limits::MAX_PAYLOAD goes: every reader takes toyos-abi's bound. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- .../a-stream-cannot-move-between-processes.md | 26 +++--- ...ves-every-byte-through-two-kernel-pipes.md | 82 ++++++++++++------- 2 files changed, 65 insertions(+), 43 deletions(-) diff --git a/issues/a-stream-cannot-move-between-processes.md b/issues/a-stream-cannot-move-between-processes.md index 6160580c7b..ecce1a6e0a 100644 --- a/issues/a-stream-cannot-move-between-processes.md +++ b/issues/a-stream-cannot-move-between-processes.md @@ -27,18 +27,20 @@ child's stdio, the handle would wait for ever or read doorbells as data, and netstack would drain what the child writes as doorbells, with no error on either side. Stage 3 makes std refuse both loudly instead. `File::from_raw_fd` of a stream's handle panics with `toyos::net`'s refusal. -Every path that moves a handle into another process answers that refusal -before anything moves: a spawn's stdio slots, `CommandExt::inherit_handle` -into any slot, `CommandExt::endow` and `Command::provide`, each starting no -child, and `toyos::ipc`'s `send_handles`, which sends nothing. And -`toyos::net` refuses by name a handle the process holds no end for, std's -`TcpStream::from_raw_fd` included. A program that hands a stream to another -process by any of these, as an inetd-style server does through its child's -stdio, then fails loudly on ToyOS where today it works, and one that takes a -moved stream back as a `TcpStream`, as socket activation does and as no -ToyOS std builds today, is refused, though root `CLAUDE.md` says existing -Rust just works. The bare `dup`, `SYS_HANDLE_SEND` and `SYS_SPAWN` syscalls -are not checked. +Every path std offers to move a raw handle into another process answers that +refusal before anything moves: a spawn's stdio slots, +`CommandExt::inherit_handle` into any slot, `CommandExt::endow` and +`Command::provide`, each starting no child. `toyos::ipc`'s `send_handles` +takes only `OwnedHandle`s, and a stream's handle becomes one only through +`OwnedHandle::from_raw`, whose contract stage 3 says excludes a handle +`toyos::net` answers for: the type refuses it. And `toyos::net` refuses by +name a handle the process holds no end for, std's `TcpStream::from_raw_fd` +included. A program that hands a stream to another process by any of these, +as an inetd-style server does through its child's stdio, then fails loudly +on ToyOS where today it works, and one that takes a moved stream back as a +`TcpStream`, as socket activation does and as no ToyOS std builds today, is +refused, though root `CLAUDE.md` says existing Rust just works. The bare +`dup`, `SYS_HANDLE_SEND` and `SYS_SPAWN` syscalls are not checked. Owner: the orchestrator, who stages `issues/netstack-moves-every-byte-through-two-kernel-pipes.md`. diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index d08b789898..8e6e8f1948 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -80,15 +80,17 @@ The design and its roast are PR #863's body. through `toyos::net`. Today a stream's handle reads and writes the stream through a `File` or a child's stdio; over the region it would read doorbells, so `File::from_raw_fd` of a stream's handle panics - with `toyos::net`'s refusal, and every path that moves a handle into - another process asks `toyos::net` of each handle before anything moves and - answers its refusal: a spawn's stdio slots, `CommandExt::inherit_handle` - into any slot, `CommandExt::endow` and `Command::provide`, each with no - child started and the handle still the caller's, and `toyos::ipc`'s - `send_handles`, which sends nothing and lets its batch go as every refused - move does, a stream's handle through `toyos::net`. So no refused move - leaves an entry that outlives its handle's number. The bare `dup`, - `SYS_HANDLE_SEND` and `SYS_SPAWN` syscalls stay unchecked. A regression, + with `toyos::net`'s refusal, and every path std offers to move a raw + handle into another process asks `toyos::net` of each handle before + anything moves and answers its refusal: a spawn's stdio slots, + `CommandExt::inherit_handle` into any slot, `CommandExt::endow` and + `Command::provide`, each with no child started and the handle still the + caller's, so no refused move leaves an entry that outlives its handle's + number. `toyos::ipc`'s `send_handles` is refused by its type: it takes + `OwnedHandle`s, and a stream's handle becomes one only through + `OwnedHandle::from_raw`, whose `# Safety` stage 3 extends to exclude a + handle `toyos::net` answers for. The bare `dup`, `SYS_HANDLE_SEND` and + `SYS_SPAWN` syscalls stay unchecked. A regression, `issues/a-stream-cannot-move-between-processes.md`. Exit: three loom models that drive the client end and netstack's ringing code as stage 3 lands them, each red under a control that reverts its decision. In one, a @@ -116,18 +118,15 @@ The design and its roast are PR #863's body. the stdio slots, `endow` and `provide` each answer the refusal and start no child, and the stream then reads and writes through that handle; each is red when its path takes the handle silently, and `endow`'s and `provide`'s - are red when the handle moves and its entry stays. `send_handles` answers - the refusal and sends nothing, and the stream's entry goes with the - handle; red when it sends, and red when the refused batch closes the handle - past `toyos::net`. A test: a `write` with room after a - reset answers `ConnectionReset` and copies nothing, red when `write` copies - before it reads the `end`. And a T14 boot of round 3's `internet_download` - on stage 3's head reads, in every run, `pipe_copy_bytes` under 0.01 per - byte downloaded, its C0, hold and spin per MB posted beside the scout's - rows. No tree that lands has that row or that counter: the boot carries the - scout's instrument, `git diff 777c96b2b 2f9e090b8` - (`wt/toyos-netring-scout` over its base), onto stage 3's head, less the - netstack stamps on the pipe writes stage 3 deletes. + are red when the handle moves and its entry stays. A test: a `write` with + room after a reset answers `ConnectionReset` and copies nothing, red when + `write` copies before it reads the `end`. And a T14 boot of round 3's + `internet_download` on stage 3's head reads, in every run, + `pipe_copy_bytes` under 0.01 per byte downloaded, its C0, hold and spin + per MB posted beside the scout's rows. No tree that lands has that row or + that counter: the boot carries the scout's instrument, `git diff + 777c96b2b 2f9e090b8` (`wt/toyos-netring-scout` over its base), onto stage + 3's head, less the netstack stamps on the pipe writes stage 3 deletes. - Stage 4: a listener and a datagram socket the same way, and the pipe ABI's messages, `SYS_CONNECTION_JOIN` and `toyos::net`'s pipe handles go. A passive child's bytes wait in [tcp]'s heap storage, at most @@ -140,17 +139,28 @@ The design and its roast are PR #863's body. listener. netstack copies each datagram record's `len`, `port` and `addr` out of the region once and judges the copy: a `len` past the bytes the record holds ends that socket alone. `toyos::net`'s `send_to` refuses - what a sender can judge and writes no record: a payload past the bound - that `toyos-abi`'s `net` module declares and [udp] reads in place of its - own `limits::MAX_PAYLOAD`, as POSIX's `sendto` fails `[EMSGSIZE]`; port 0; - an unspecified address. [udp] still refuses each by its counter at the + what a sender can judge and writes no record: a payload past + `toyos_abi::net::UDP_MAX_PAYLOAD`, as POSIX's `sendto` fails + `[EMSGSIZE]`; port 0; an unspecified address. That bound is the one + declaration: [udp]'s `limits::MAX_PAYLOAD` goes, and [udp], the node's + `name.rs` and `resolve.rs`, netstack's `serve.rs` and [udp]'s + `tests/send.rs` read it. [udp] still refuses each by its counter at the trust boundary. What only netstack can judge, a connected socket's other destination, a route, broadcast permission, a source address and a pending - ICMP error, netstack stores in the region's `pending` word and rings, and - the socket's next call answers it and clears it, as POSIX's pending error - is "returned in response to a subsequent send(), recv(), or getsockopt() - operation" (XSH 2.10.10). netstack takes a record only while [udp]'s send - queue has room. Exit: over the region, + ICMP error, netstack stores in the region's `pending` word if it is 0, + then exchanges both client wants for 0 and rings each that was set, as it + does for a stream's fatal `end`; a call's look after it arms reads + `pending` beside its ring, and the socket's next call answers it and + clears it, as POSIX's pending error is "returned in response to a + subsequent send(), recv(), or getsockopt() operation" (XSH 2.10.10). + netstack takes a record off the send ring only while [udp]'s queue for + the socket has room, and looks again when a datagram leaves it, so a full + queue holds the ring and refuses nothing. A `send_to` over a full send + ring waits as a stream's writer does, on a bell of the socket's own that + netstack makes at the bind and alone writes, its want `tx_room`, as + POSIX's `sendto` "shall block until space is available or a timeout + occurs"; nonblocking, which std's `UdpSocket::set_nonblocking` then sets, + it answers `WouldBlock`. Exit: over the region, `system/netstack/node/tests/listeners.rs`'s `a_listener_answers_a_syn_and_wakes_its_owner_when_the_handshake_ends` passes, and so does a case in which a child holds, before its accept, text @@ -163,7 +173,17 @@ The design and its roast are PR #863's body. that `len` as written. A case per refusal: an oversized payload, port 0 and an unspecified address each answer their error at `send_to` with no record written, and each refusal only netstack judges answers on the - socket's next call; each is red when that call answers `Ok`. + socket's next call; each is red when that call answers `Ok`. Two loom + models drive the client end and netstack's code as stage 4 lands them. In + one, a `recv_from` asleep with nothing to receive, and a `send_to` asleep + over a full send ring, each answers a refusal only netstack judges that + lands meanwhile; each is red when the look reads `pending` before it arms, + red when netstack exchanges the wants before it stores `pending`, and red + under `want-relaxed`, and the sender's is red when netstack rings only + `rx_data`. In the other, a sender that fills the send ring while [udp]'s + queue is full, beside a `recv_from` asleep on the connection, has every + record sent, in order, none refused; red when netstack refuses a record at + the full queue, and red when it rings `tx_room` on the connection. Owner: the orchestrator, who stages it. From 67a1d5963e53743971b57cc58cc6bf8510226cb8 Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 04:09:02 +0200 Subject: [PATCH 13/14] issues: the datagram room model holds a record at a full queue, and a nonblocking send at a full ring answers WouldBlock Round 8's BLOCKER on #863: the room model's datagram left netstack's queue before netstack's first look, so no interleaving had netstack hold a record at a full queue, and a netstack that waited for the client's next ring after the leave stayed green. Stage 4's exit now has the leave as its own step, interleaved with netstack's looks, and a control that waits for the client's ring after the leave without looking again; and names `want-relaxed`, which the record claimed red. Round 8's NOTE: a case for a nonblocking `send_to` over a full send ring, red when `set_nonblocking` sets nothing. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...-moves-every-byte-through-two-kernel-pipes.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 8e6e8f1948..8eca384e5e 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -173,17 +173,23 @@ The design and its roast are PR #863's body. that `len` as written. A case per refusal: an oversized payload, port 0 and an unspecified address each answer their error at `send_to` with no record written, and each refusal only netstack judges answers on the - socket's next call; each is red when that call answers `Ok`. Two loom + socket's next call; each is red when that call answers `Ok`. A case: a + nonblocking `send_to` over a full send ring answers `WouldBlock` and + writes no record, red when `set_nonblocking` sets nothing. Two loom models drive the client end and netstack's code as stage 4 lands them. In one, a `recv_from` asleep with nothing to receive, and a `send_to` asleep over a full send ring, each answers a refusal only netstack judges that lands meanwhile; each is red when the look reads `pending` before it arms, red when netstack exchanges the wants before it stores `pending`, and red under `want-relaxed`, and the sender's is red when netstack rings only - `rx_data`. In the other, a sender that fills the send ring while [udp]'s - queue is full, beside a `recv_from` asleep on the connection, has every - record sent, in order, none refused; red when netstack refuses a record at - the full queue, and red when it rings `tx_room` on the connection. + `rx_data`. In the other, [udp]'s queue starts full and a datagram leaves + it as its own step, interleaved with netstack's looks, so netstack can find + a published record while the queue is full and hold it; a sender that + fills the send ring, beside a `recv_from` asleep on the connection, has + every record sent, in order, none refused. It is red when netstack refuses + a record at the full queue, red when it rings `tx_room` on the connection, + red when after the leave it waits for the client's next ring without + looking at the send ring again, and red under `want-relaxed`. Owner: the orchestrator, who stages it. From 6423e5928e449266aa79fc3bb7543accbd22c137 Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 04:50:15 +0200 Subject: [PATCH 14/14] issues: a held send ring is looked at again when a datagram leaves the queue or is dropped [udp]'s queue empties by two paths: a datagram leaves when Udp::serve offers it and the caller takes it, and one whose hop Udp::fail names is dropped. The track named only the first, and no exit case could fail if stage 4 hooked the second look to the transmit path alone. Stage 4's exit gains a node case over fail: a full queue to a silent hop, a sender asleep over a full send ring, woken when [ip] gives the hop up. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...ves-every-byte-through-two-kernel-pipes.md | 21 ++++++++++++------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 8eca384e5e..8714da2b72 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -154,13 +154,14 @@ The design and its roast are PR #863's body. clears it, as POSIX's pending error is "returned in response to a subsequent send(), recv(), or getsockopt() operation" (XSH 2.10.10). netstack takes a record off the send ring only while [udp]'s queue for - the socket has room, and looks again when a datagram leaves it, so a full - queue holds the ring and refuses nothing. A `send_to` over a full send - ring waits as a stream's writer does, on a bell of the socket's own that - netstack makes at the bind and alone writes, its want `tx_room`, as - POSIX's `sendto` "shall block until space is available or a timeout - occurs"; nonblocking, which std's `UdpSocket::set_nonblocking` then sets, - it answers `WouldBlock`. Exit: over the region, + the socket has room, and looks again when a datagram leaves it or is + dropped, so a full queue holds the ring and refuses nothing. A + `send_to` over a full send ring waits as a stream's writer does, on a + bell of the socket's own that netstack makes at the bind and alone + writes, its want `tx_room`, as POSIX's `sendto` "shall block until space + is available or a timeout occurs"; nonblocking, which std's + `UdpSocket::set_nonblocking` then sets, it answers `WouldBlock`. Exit: + over the region, `system/netstack/node/tests/listeners.rs`'s `a_listener_answers_a_syn_and_wakes_its_owner_when_the_handshake_ends` passes, and so does a case in which a child holds, before its accept, text @@ -175,7 +176,11 @@ The design and its roast are PR #863's body. record written, and each refusal only netstack judges answers on the socket's next call; each is red when that call answers `Ok`. A case: a nonblocking `send_to` over a full send ring answers `WouldBlock` and - writes no record, red when `set_nonblocking` sets nothing. Two loom + writes no record, red when `set_nonblocking` sets nothing. A case: the + socket's queue is full to a next hop that never answers and a `send_to` + sleeps over a full send ring; when [ip] gives the hop up and [udp]'s + `Udp::fail` drops the queue, netstack takes the held record and the sender + wakes, red when netstack looks again only after a datagram leaves. Two loom models drive the client end and netstack's code as stage 4 lands them. In one, a `recv_from` asleep with nothing to receive, and a `send_to` asleep over a full send ring, each answers a refusal only netstack judges that