ToyOS's TCP offers a scaled window that grows by the receiver's own round trip, learned from the peer's own segment size, and RFC 8985's loss probe keeps a lost window update from stalling a sender

[Japabu]

This branch merged origin/main last at 49dca4e02 (which carries #810, #816 and #823). Its own change is d286ece79 (round 1), cd34232c9 (round 2), 6df8222c1 (round 3), a700899ee (round 4) and e7efa0a87 (round 5, which deletes round 4's production change: conn.rs is again 6df8222c1's, and against round 3 the branch adds only one test, recovery.rs +20/−1): git diff origin/main...49dca4e02, 13 files, +824/−67. The figures that follow are round 3's. Production (toyos-net-shard/tcp/src without the cfg(test) props.rs and rx.rs's test module, plus netstack): +294/−44. Tests: +512/−24.

What changed and why

toyos-net-tcp already negotiated RFC 7323 window scaling (both ways, never unless both SYNs offer it, the shift clamped at 14 and counted WscaleClamped) and timestamps with PAWS. It took the smallest shift that fits the receive buffer, and netstack's TCP_BUFFER of 65,535 made that shift 0. So the work is the buffer, what its growth needs, and what a sender needs once windows reopen by update.

• netstack's TCP_BUFFER is 4 MiB each way, window scale 7. At 1 Gb/s that covers a 33.5 ms round trip; Ubuntu on the T14 autotunes to 6 MB (tcp_rmem max). PLACE_BYTES becomes a stream's: two 2 MiB pipes and two 4 MiB buffers, 12 MiB, up from the listener's 10 MiB: about 170 places on 16 GiB instead of about 200, under the same one-eighth share.
• The receive buffer grows per connection, only by reads (rx's module doc). It starts at limits::RECEIVE_BUFFER_INITIAL = 65,535, the most a SYN offers. Once a round trip has passed it grows to twice what the user read per round trip, up to the configured buffer, and never shrinks (RFC 7323 §2.4). Text the toyos-net-tcp user does not read grows nothing. At netstack this does not bound a client that stops reading: sockets.bridge drains a connection into its client's 2 MiB pipe whether or not the client reads, so such a connection's buffer can still grow toward 4 MiB; it stays inside PLACE_BYTES. A window field at the negotiated shift bounds the growth, so an unscaled connection stays at 65,535.
• The round trip growth uses is the receiver's own. A downloader sends no data, so the sender-side SRTT keeps the handshake's sample however the path's delay moves. With timestamps, a full-sized in-order segment carrying a TSecr not yet seen samples the age of that echo, averaged with gain 1/8 (Linux's tcp_rcv_rtt_measure_ts); without them, the least time the peer took to fill the window offered (Linux's tcp_rcv_rtt_measure).
• Full-sized is learned from what arrives (round 3, Linux's tcp_measure_rcv_mss): a segment at least as long as the largest so far raises it, up to what our SYN offered less the timestamp option; two in a row of one shorter length, not under the floor MSS less the option, lower it. Round 2 measured against this end's send MSS, which is the T14 regression below.
• An echo's age is at least one tick and at most twice the estimate (round 3). One tick is Linux's floor (tcp_rtt_tsopt_us). The ceiling is ours: the echo of the last ACK before the peer fell silent ages by the silence, and now moves the estimate by an eighth of itself, not by an eighth of the silence; a real rise is followed at ×1.125 per new echo, about one a millisecond in a transfer.
• Ring storage grows with what is held (doubling, laid out again from the head), so a 4 MiB send buffer costs what it holds.
• After a read, a window update leaves when the window could at least double (Linux's tcp_cleanup_rbuf rule). netstack transmits a received batch's ACKs before sockets.bridge drains the batch; without the update the sender learned of the room one round trip late and the growth rule settled at about 133 KB.
• A lost window update no longer stalls a sender: RFC 8985 §7's tail loss probe. The update that reopens a window-limited round can be the tail's only ACK; with every second ACK lost at 65,535 (s_net_005) the sender sat until the RTO, cwnd fell to one segment, and 1 MiB had not arrived at t = 582,893 ms. Now, with SACK, two SRTTs after the last new data or advancing ACK (plus WCDelAckT, 200 ms, with one segment out; plus Linux's TCP_TIMEOUT_MIN, 2 ms, otherwise; never later than the RTO, which it then stands in for once), the sender sends a segment of new data where the peer's window takes a whole one, ignoring cwnd, and otherwise the last segment again (§7.3). A duplicate ACK or new SACK information cancels it. Counters tcp.loss-probe and tcp.loss-probe-recovery. This is RFC 8985's TLP without RACK: loss below the tail is still found by RFC 6675. Round 3 brings it to the RFC's text:
  • §7.4.2: at or past the probe's end, a probe of new data, a D-SACK whose right edge is the probe's end, or a duplicate ACK without SACK ends the episode with nothing lost; only an ACK past the end without either says a resent probe repaired a loss and reduces cwnd once (Linux's tcp_process_tlp_ack). An ACK at the end leaves the episode open, since the original's ACK reads the same. tx::read_sack now returns the D-SACK's right edge.
  • §7.3: no probe is scheduled without an RTT sample since the last probe left.
  • §7.2: none in RTO recovery, that is until SND.UNA passes the recover point the expiry set.
  • §7.1: entering fast recovery clears the probe in flight and a due one, and arms the RTO if the probe had taken it.
  • One ACK that infers the probe's loss and enters SACK recovery cuts twice, as Linux v6.12 does (round 5). Round 4 added a flag so that recovery entry skipped its reduction on such an ACK, on the premise that Linux cuts once there; the premise was false and round 5 deletes the flag. In Linux v6.12, tcp_ack runs tcp_process_tlp_ack (tcp_input.c:4038) before tcp_fastretrans_alert (:4048); its loss branch (:3846-3849) runs tcp_init_cwnd_reduction, enters CWR, runs tcp_end_cwnd_reduction and calls tcp_try_keep_open (:2760-2772), which leaves CWR for Open or Disorder; so tcp_enter_recovery (:2897) finds tcp_in_cwnd_reduction (tcp.h:1333, CWR or Recovery only) false and runs tcp_init_cwnd_reduction again (:2915-2919). Read from tcp_input.c at tag v6.12, sha256 8007c66e…5d859749d. The flag also guarded one ACK, not one congestion event: the probe's reduction sets no recover, so an ACK later in the same window that entered recovery cut again anyway.
  • A probe that came due but had not left (its next hop not ready) gives way when an ACK shuts the window: persist alone runs.
• A scaled window that would round to a zero field is offered as one unit where the buffer has room (as Linux's __tcp_select_window). With a hole open the edge holds; at shift 7 a 93-byte window read as zero and a property run stalled 600 s.
• Info.rcv_capacity, Info.rcv_rtt and Rx::rtt() are gone (round 3): nothing shipping read them. Rx::capacity() is cfg(test), read by props.rs's capacity invariant alone. The tests observe the window through rcv_edge − rcv_nxt, and the estimator through rx's own unit tests.
• Timestamps: already negotiated, with PAWS. At 1 Gb/s the 2^31 sequence half-space wraps in about 17 s, inside TIME-WAIT, so PAWS is what keeps an old duplicate out at these windows.

The T14 regression at round 2, and its cause

The orchestrator's T14 reading at 0350817cd (round 2 plus the measurement commit): 24.1 Mb/s, the download cut at the window after 120 MB, where round 1's head did 328.2 Mb/s on the same machine and URL. 24.1 Mb/s is under the 65,535 ceiling at its rtt_ms (65,535 · 8 / 15.0 ms = 35.0 Mb/s), and every CPU sat under 5% busy: a window that never grew, not a CPU bound. (Its 31.9 ms of CPU per MB is idle overhead spread over a slow transfer.)

Reproduced in the crate's host network at the T14's shape: node 1 sends 128 MiB to node 0 at 125,000 bytes per ms of transmit credit, a 15 ms round trip, timestamps and SACK both ways, 4 MiB buffers; node 0's SYN rewritten to offer an MSS of 1,460, 1,440 or 1,392, so node 1's segments are 1,448, 1,428 or 1,380 bytes against node 0's own send MSS of 1,448. The test file is repro-t14.rs beside the logs; same file, each crate copied whole at its head, cargo test --release --test t14, each EXIT=0:

peer's segments	round 1 d286ece79	round 2 cd34232c9	round 3 6df8222c1
1,448, timestamps	1222.9 Mb/s, window 4 MiB	1222.9, 4 MiB	1222.9, 4 MiB
1,428, timestamps	1202.4, 4 MiB	34.2, window 65,535	1202.4, 4 MiB
1,380, timestamps	1202.4, 4 MiB	34.3, window 65,535	1202.4, 4 MiB
1,448 and 1,380, no timestamps	1222.9 and 1202.4	1222.9 and 1244.2	1222.9 and 1244.2

(The harness's credit is per millisecond with no queue, so its rate reads above the gigabit; the rows compare crates, not links.) The cause is round 2's echo sampler: it took a sample only from a segment at least smss() long, this end's send MSS. A timestamped peer whose segments are shorter, by an MSS it clamps lower than ours or a path it sizes for, was never sampled, rcv_rtt stayed None, and the buffer stayed at 65,535 for the connection's life. The peer of the T14's URL is CloudFront; probed from this development machine on the same day, it negotiates timestamps, SACK and window scale and offers an MSS of 1,440 (macOS's TCP_CONNECTION_INFO: maxseg 1,428). Its segments' size on the T14's path is not read here; the measurement commit now prints the receiver's learned rcv_mss to read it. The other two candidates are ruled out: the loss probe is a sender's mechanism and the downloader sends only its request (server rto 0 and no retransmitted byte in every row); the measurement commit changes nothing in the stack and the regression reproduces without it.

Tests and their controls

Host tests, ours against ours (the crate's consistency control):

• a_peer_sending_segments_shorter_than_our_send_mss_still_grows_the_window (new, net.rs): the T14 shape above with 1,380-byte segments, 32 MiB; asserts the peer's longest segment is 1,380, both shifts 7, and the receiver's window reaches the whole 4 MiB.
• rfc_8985_7_4_a_resent_probe_acknowledged_past_its_end_without_a_dsack_repaired_a_loss (replaces round 2's …acknowledged_without_a_dsack…, which ACKed exactly the end): the ACK at the end leaves the episode open and reduces nothing; the ACK past it reduces cwnd once. rfc_8985_7_4_a_dsack_after_the_ack_at_the_probes_end_infers_no_loss (the review's): ACK at the end, then the D-SACK, then an ACK past the end: tcp.loss-probe-recovery 0, cwnd stands. rfc_8985_7_4_a_dsack_of_another_segment_still_infers_the_loss and rfc_8985_7_4_a_duplicate_without_sack_at_the_probes_end_infers_no_loss (Case 2).
• rfc_8985_7_3_no_probe_without_an_rtt_sample_since_the_last (the review's, net.rs): no timestamps, SACK, a 20 ms handshake and then a 100 ms path, ten whole segments a round, 150 ms between rounds so the last probe's D-SACK has ended its episode. Probes leave in rounds 1, 3, 5 and 7, every other round, then none; SRTT reaches 99.3 ms. Asserted: no two rounds in a row probe, none after round 20, SRTT in [90, 110] ms, no RTO, no inferred loss. The review asked for at most one probe: under RFC 6298 one sample a round moves SRTT an eighth of the way, so SRTT needs four samples to pass RTT/2, and §7.3 permits a probe in each round that follows one. Without the condition (mutation s1) a probe leaves every round and SRTT stays at 20 ms.
• rfc_8985_7_4_an_ack_that_infers_the_probes_loss_and_enters_recovery_cuts_twice (round 3's NOTE, its assertion corrected in round 5): ten out, the probe resends 14033, five more go out on the ACK at its end, then one ACK of 16929 SACKing 18377–22721: one loss-probe-recovery, one sack-recovery, ssthresh and cwnd 4,054, the second cut's, from the 4 × 1,448 in flight after the ACK. Its oracle is the Linux sequence above: two reductions on the one ACK, the probe's then recovery's.
• rfc_8985_7_2_no_probe_in_rto_recovery (the review's): s_lr_019's timeline, then a plain ack(2449) at 232 ms and time to past 2·SRTT + 2 ms and to just before the RTO: one probe, one RTO. rfc_8985_7_1_fast_recovery_ends_the_probes_episode: the resent probe outstanding as SACK recovery begins; recovery's own reduction is the only one. a_probe_due_when_the_window_shuts_gives_way_to_persist (the NOTE): the PTO fires with the next hop pending, an ACK shuts the window, the hop wakes: nothing leaves, no probe counted.
• rx's unit tests (new): full_sized_is_learned_from_what_arrives (1,380 sampled; one 1,380 after a 1,448 not, the second in a row lowers full-sized and is), an_echo_aged_by_the_peers_silence_moves_the_estimate_an_eighth (16 ms, then a 10 s echo moves it to 18 ms, then a 0 ms echo counts as 1 ms), without_timestamps_the_least_fill_time_is_kept.
• the_receive_window_grows_by_what_is_read_per_round_trip (round 2's, now without rcv_rtt): handshake at 20 ms, then an 80 ms path; the downloader's SRTT asserted to stay the handshake's; the reader takes 4 KB/ms in two bursts every 20 ms. Asserts, with timestamps and without, that the window never exceeds 2·(R·120 ms + burst) + one MSS, an estimate at most half again the path's, and that over the last 2 s the reader got exactly its rate, which an estimate under the path's would not give.
• Rounds 1 and 2's: s_net_005_lost_acks at 65,535 and 4 MiB, each drop parity; a_sub_unit_window_rounds_up_only_into_free_room and props.rs's capacity invariant; rfc_8985_7_3_the_probe_is_new_data…, rfc_8985_7_4_…reported_as_a_duplicate…, rfc_8985_7_2_with_one_segment_out…; s_lr_018/s_lr_019 meeting the probe at 22 ms before their RTO at 222 ms; rfc_7323_2_…in_flight_each_way, rfc_7323_2_2_no_scaling_unless_both_syns_offer_it, a_receive_buffer_nobody_reads_never_grows.

Negative control. Production src/ at round 2's head cd34232c9 under this round's tests/: EXIT=101, nine reds, every new or changed scripted and network test above (control-round2-src.log).

Mutations, round 3, each a checked patch on a copy of the crate at 6df8222c1, cargo test --offline --no-fail-fast, git apply -R; every one EXIT=101, RESTORED=0, the copy clean; the unmutated copy EXIT=0, 405 passed. Patches in the comment of this round. a2, a3 and g4 stayed green in a first run, and s1 and s4 did against the first form of their tests; the tests above were added or changed for them and the whole set run again.

mutation	reds
a1 loss inferred at the probe's end (the review's)	the four rfc_8985_7_4_… above, rfc_8985_7_3_no_probe…
a2 any D-SACK ends the episode	rfc_8985_7_4_a_dsack_of_another_segment…
a3 no duplicate-ACK case	rfc_8985_7_4_a_duplicate_without_sack…
s1 no RTT-sample condition	rfc_8985_7_3_no_probe_without_an_rtt_sample…
s2 probes in RTO recovery	rfc_8985_7_2_no_probe_in_rto_recovery
s3 probe kept into fast recovery	rfc_8985_7_1_fast_recovery_ends…
s4 due probe kept into persist	a_probe_due_when_the_window_shuts…
m1 full-sized is our MSS (round 2's rule)	full_sized_is_learned…, a_peer_sending_segments_shorter…
m2 full-sized never lowered	full_sized_is_learned…
e1 echo unclamped; e2 a zero age taken	an_echo_aged_by_the_peers_silence…
g1 growth ×64	the_receive_window_grows…
g2 grown on every read	the_receive_window_grows…, a_peer_sending_segments_shorter…, rfc_7323_2_…in_flight_each_way
g3 no echo sample	two rx tests and four network tests on growth
g4 fill keeps the largest	without_timestamps_the_least_fill_time_is_kept
r1 round-up's unit <= free deleted	a_sub_unit_window…, 12 property tests through the capacity invariant
p1 no loss probe	s_net_005_lost_acks, every rfc_8985_* but 7_3_no_probe…, s_lr_018, s_lr_019
p2 no loss response	rfc_8985_7_4_…past_its_end…, rfc_8985_7_4_a_dsack_of_another…
p3 D-SACK ignored	s_net_005_lost_acks, rfc_8985_7_4_a_dsack_after…, rfc_8985_7_3_no_probe…
p4 no WCDelAckT	rfc_8985_7_2_with_one_segment_out…, s_rt_010_a_retransmission_echoed

Mutation, round 5, cut1: round 4's cut flag re-applied at 49dca4e02 as a checked patch (git diff 6df8222c1 a700899ee -- toyos-net-shard/tcp/src/conn.rs, posted in this round's comment). git apply --check, applied, cargo test --offline --no-fail-fast in the crate: EXIT=101, only …enters_recovery_cuts_twice red, left: (5068, 5068), right: (4054, 4054); git apply -R RESTORED=0, git status --porcelain empty (r5/logs/mut-cut-once.log).

Round 1's m1–m8 and its whole-change revert are in the first mutation comment and stand for d286ece79's part.

Independent oracle. RFC 8985 §7's text (§7.1–§7.4.2's pseudocode) and RFC 7323; Linux v6.12's tcp_input.c (tcp_process_tlp_ack, tcp_try_keep_open, tcp_enter_recovery, tcp_measure_rcv_mss, tcp_rcv_rtt_measure_ts, tcp_rtt_tsopt_us) and tcp_output.c (tcp_schedule_loss_probe, tcp_send_loss_probe) as read; and the T14 against a CDN for the scaled path. The host's TCP cannot be the peer without a TAP device and privileges, and QEMU's slirp never offers Window Scale (tcp_dooptions parses MSS alone). A smoltcp peer, which round 1's review proposed, is declined: the owner ruled "no more smoltcp. fast track make it gone", and #801 removed it.

No new guest test. A QEMU guest's peer is slirp, which never scales, never loses an ACK and never sends short segments on its own; the probe, the growth and the segment-size learning are reached by the host network's impairment, rewriting and clock, which a guest cannot drive. The whole guest suite checks the no-offer path against a third-party TCP.

Gates at 49dca4e02 (round 5)

Logs under the orchestrator's wscale/r5/logs/, each from one script run on the committed head: its first line head 49dca4e02…, its second the command, its last line the command's own EXIT=. The worktree's git status --porcelain --ignore-submodules=none was empty after the last (gates.done).

• cargo test --offline --no-fail-fast in toyos-net-shard/tcp: EXIT=0, 406 passed (crate-tests.log).
• cargo run -- --ci host: EXIT=0, [ci] Host: 78 step(s), all green (ci-host.log).
• cargo run -- --build-only: EXIT=0 (build-only.log).
• Guest suite, cargo test: EXIT=0, 44 passed, 44 total (and the harness's own 348 passed, 15 ignored), with netstack_streams, netstack_streams_e1000e, netstack_socket_churn and libc_sockets all PASS; host load averages 66.92 52.41 52.41 before, 52.17 53.40 52.93 after (guest.log).
• T14: round 3's reading below stands for 49dca4e02. git diff 6df8222c1 49dca4e02 -- toyos-net-shard userland/netstack is recovery.rs alone, so the production code the reading ran is byte-identical. Round 4's a700899ee and its gates are superseded: its production change is deleted.
• Round 4's red-before.log and crate-tests.log are no longer cited. The first had no EXIT line, and the second had neither an EXIT line nor the head.

Gates at 6df8222c1 (round 3)

• cargo test --offline --no-fail-fast in toyos-net-shard/tcp: EXIT=0, 405 passed (tcp-tests.log).
• cargo run -- --ci host: EXIT=0, [ci] Host: 78 step(s), all green (ci-host.log).
• cargo run -- --build-only: EXIT=0 (build-only.log).
• Guest suite, cargo test: EXIT=0, 43 passed, 43 total, netstack_streams, netstack_streams_e1000e, netstack_socket_churn, libc_sockets among them; host load averages 66.37 before, 64.66 after (guest.log).
• T14: staged on the measurement branch wt/toyos-wscale-metal at 30c6678ec (this head merged at e1b14a5e4 onto round 2's measurement branch, plus one measurement-only commit that never lands: the download twice back to back with cpu_s and cpu_ms_per_mb per run, and netstack saying each connection's rcv_shift, rcv_capacity, rcv_wnd, rcv_rtt, rcv_mss, srtt, cwnd and the stack's recovery counts, on the client's close and now also when the client goes without one, which is how round 2's cut run said nothing). cargo test --test toyos-build -- --metal --metal-readback <dir> internet_download: EXIT=2, staged, the machine not touched; image sha256 e2c09588…36b8a6fb. The orchestrator's reading (PR comment): judge EXIT=0, PASS internet_download; first download 107.1 Mb/s, second 662.4 Mb/s, both rcv_shift=7 rcv_capacity=4194304 rcv_mss=1424, loss_probe=0 retransmit_bytes=0; rtt_ms 15.0–17.8. Round 2's 24.1 Mb/s regression is gone.

What I am unsure of

• The T14 read the CDN's segments as 1,424 bytes (rcv_mss=1424), shorter than our send MSS less the option, the shape the host reproduction showed round 2 never sampled.
• On the ACK that cuts twice, the sequence is Linux's but the values are not: our on_loss takes ssthresh from FlightSize (RFC 5681 §3.2, RFC 6675), so the second cut is 0.7 × the 5,792 in flight after the ACK = 4,054, where Linux's CUBIC (tcp_cubic.c:341-356) takes it from cwnd in segments, which the first cut has already lowered. The test asserts the count and order of the cuts against Linux; its value is our FlightSize rule's, not compared with Linux's.
• The echo ceiling of twice the estimate is ours, not Linux's (Linux takes the silence's whole age at gain 1/8); it bounds an echo aged by a silence, and follows a real rise at ×1.125 per new echo.
• The loss probe is TLP without RACK: a probe of new data SACKed above a lost tail segment does not mark that segment lost, so a real single-segment tail loss with new data to send waits for the RTO after the probe, as before the probe.
• The 2 ms slack and WCDelAckT of 200 ms are Linux's and RFC 8985's values, not measured here.
• Toward Ubuntu's ~800 Mb/s warm on the T14, outside this fence and by reading, not measurement: the I219's RX_RING of 256 × 2 KiB descriptors is about 3.1 ms of gigabit frames before missed; netstack takes RX_BUDGET 64 frames a pass; TX_RING is 16. Round 1 used about 13.7 ms of CPU per MB where Ubuntu uses about 2.

🤖 Generated with Claude Code

https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C

[Japabu]

Mutation patches at d286ece79, each applied with git apply, cargo test --offline --no-fail-fast in toyos-net-shard/tcp, then git apply -R (RESTORED=0, tree clean). Every one EXIT=101; which tests each reds is in the body's table.

m1-no-scale-offered

diff --git a/toyos-net-shard/tcp/src/open.rs b/toyos-net-shard/tcp/src/open.rs
index 7322f8503..af20ede15 100644
--- a/toyos-net-shard/tcp/src/open.rs
+++ b/toyos-net-shard/tcp/src/open.rs
@@ -160,7 +160,7 @@ fn syn_options(local: &Local, peer: Option<&Negotiated>, now: Instant) -> SynOpt
         mss: Some(local.mss),
         sack_permitted: peer.is_none_or(|n| n.sack),
         timestamps,
-        window_scale: if peer.is_none_or(|n| n.scaled) { WindowShift::new(local.shift).ok() } else { None },
+        window_scale: None,
     }
 }
 

m2-absent-option-ignored

diff --git a/toyos-net-shard/tcp/src/open.rs b/toyos-net-shard/tcp/src/open.rs
index 7322f8503..278ba060c 100644
--- a/toyos-net-shard/tcp/src/open.rs
+++ b/toyos-net-shard/tcp/src/open.rs
@@ -38,7 +38,7 @@ pub fn negotiate(seg: &In<'_>, local: &Local, first_tsval: u32, ctx: &mut Ctx<'_
         }
         Some(mss) => mss,
     };
-    let scaled = options.window_scale().is_some();
+    let scaled = true;
     let (snd_shift, rcv_shift) = match options.window_scale() {
         Some(scale) => {
             if scale.raw() > scale.effective() {
@@ -46,7 +46,7 @@ pub fn negotiate(seg: &In<'_>, local: &Local, first_tsval: u32, ctx: &mut Ctx<'_
             }
             (scale.effective(), local.shift)
         }
-        None => (0, 0),
+        None => (0, local.shift),
     };
     let ts = options.timestamps().map(|t| Ts { recent: t.value, recent_at: ctx.now, offset: local.ts_offset, first: first_tsval });
     Negotiated { peer_mss, snd_shift, rcv_shift, scaled, sack: options.sack_permitted(), ts }

m3-no-growth

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index a23e09cc2..a12195bc7 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -392,7 +392,7 @@ impl Rx {
             let read = u128::try_from(read).unwrap_or(u128::MAX);
             let per_rtt = read.saturating_mul(rtt.as_nanos()).checked_div(now.since(began).as_nanos()).unwrap_or(0);
             let want = usize::try_from(per_rtt.saturating_mul(2)).unwrap_or(usize::MAX);
-            self.buf.grow(want.min(self.max));
+            let _ = want;
             self.round = (now, 0);
         }
         if let Some(candidate) = self.candidate(mss) {

m4-fixed-full-buffer

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index a23e09cc2..54e3b37b2 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -90,7 +90,7 @@ impl Rx {
             next,
             edge: next.add(window),
             shift,
-            buf: Ring::new(max.min(initial)),
+            buf: Ring::new(max),
             ranges: Vec::new(),
             stamp: 0,
             fin: None,

m5-no-window-update

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index a23e09cc2..1dd7f64cc 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -396,7 +396,7 @@ impl Rx {
             self.round = (now, 0);
         }
         if let Some(candidate) = self.candidate(mss) {
-            if self.last_window < self.threshold(mss) || candidate.since(self.next) >= self.window().saturating_mul(2) {
+            if self.last_window < self.threshold(mss) {
                 self.ack_now = true;
             }
         }

m6-no-round-up

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index a23e09cc2..a60e8a25b 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -274,7 +274,7 @@ impl Rx {
         // room, as Linux does: else a sender owing the text of a hole waits on a window not shut.
         let unit = 1u32.checked_shl(u32::from(self.shift)).unwrap_or(u32::MAX);
         let window = edge.since(self.next);
-        let edge = if window > 0 && window < unit && unit <= self.free() { self.next.add(unit) } else { edge };
+        let _ = (window, unit);
         let field = (edge.since(self.next) >> self.shift).min(u32::from(u16::MAX));
         (edge, u16::try_from(field).unwrap_or(u16::MAX))
     }

m7-no-relayout

diff --git a/toyos-net-shard/tcp/src/ring.rs b/toyos-net-shard/tcp/src/ring.rs
index aaf45daab..e47345fce 100644
--- a/toyos-net-shard/tcp/src/ring.rs
+++ b/toyos-net-shard/tcp/src/ring.rs
@@ -53,7 +53,7 @@ impl Ring {
     /// offset, laid out again from the head.
     fn reserve(&mut self, end: usize) {
         if end > self.bytes.len() {
-            self.bytes.rotate_left(self.head);
+            let _ = 0;
             self.head = 0;
             let len = end.max(self.bytes.len().saturating_mul(2)).min(self.capacity);
             self.bytes.resize(len, 0);

m8-no-offerable-cap

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index a23e09cc2..65693a591 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -85,7 +85,7 @@ impl Rx {
     pub fn new(next: Seq, max: usize, shift: u8, window: u32, now: Instant) -> Self {
         let initial = usize::try_from(crate::limits::RECEIVE_BUFFER_INITIAL).unwrap_or(usize::MAX);
         let offerable = usize::from(u16::MAX).checked_shl(u32::from(shift)).unwrap_or(usize::MAX);
-        let max = max.min(offerable);
+        let _ = offerable;
         Self {
             next,
             edge: next.add(window),

[Japabu]

T14 at 751dfc6f6 (wt/toyos-wscale-metal: this branch's d286ece79 merged with #818's row), run by the orchestrator: download boot, image sha256 e35e2466…cb658539e checked against request.txt, toyos-metal --fat32-check exit 0. Judge cargo test --test toyos-build -- --metal --metal-readback <dir> internet_download: EXIT=0, PASS internet_download. The reading: [download] whole bytes=170439044 sha256=1a9ee8ca…f9d00e7f secs=4.154 mbps=328.2 rtt_ms=[16.9 15.2 15.5 16.0 16.0] busy=0.070 cpus=[0.024 0.002 0.187 0.002 0.002 0.002 0.339 0.006]. Against the 64 KiB ceiling (524.3 / 15.2 = 34.5 Mb/s): 9.5x, so #818's issue's shift arm is met on hardware. Against #818's reading at the old buffer (35.3 Mb/s): 9.3x. Against Ubuntu on the same machine, cable and URL an hour earlier (orchestrator's curl with the project User-Agent: 340.8 Mb/s cold, then 827.2 and 803.6): about 40% of Ubuntu's warm rate. Busiest CPUs 33.9% and 18.7%. One boot.

[Japabu]

Review of #820 at d286ece79 (own change d286ece79~1..d286ece79; the #801 part is reviewed on #801).

Net lines, own change: production +102/−37 (net +65: rx.rs, ring.rs, conn.rs, stack.rs, lib.rs, netstack main.rs), tests +114/−4. Whole branch against origin/main with #801: 74 files, +2608/−4033.

BLOCKER

• toyos-net-shard/tcp/tests/net.rs:327 — s_net_005_lost_acks was moved from 65,535 to 4 MiB because the new update rule turns it red at 65,535. The red is a regression any unscaled peer reaches. With a peer that sends no Window Scale (slirp, or any path that strips the option), Rx::new clamps max to 65,535 at shift 0, which is exactly the state that reds. The branch's own logs (s005-new-0.log, s005-new-1.log) show more than "an RTO each time": at t = 582,893 ms the 1 MiB transfer has not finished (720,031 sent, srtt 43 s, rto 60 s, cwnd 2,896), where the base finished it with zero retransmitted bytes. At 4 MiB a 1 MiB transfer is never window-limited, so the test no longer covers the window-limited case it guarded. That is a weaker check bought to pass, and nothing in issues/ records the regression. Restore a 65,535 arm of s_net_005 beside the 4 MiB one. Then fix the window-limited reopen so that one lost update ACK does not stall the sender (a tail-loss probe, or an update rule that does not leave the reopen on a single ACK), or file it in issues/ with an owner, these logs as evidence and that arm as the exit. Root CLAUDE.md makes a red test a defect: fix it, or delete it while its issue records the commit that restores it.

• PR body, "Gates", T14 — the change exists for the T14's bandwidth, and the only independent oracle the body names for the scaled path is the T14 against a CDN. That reading is staged (metal-stage.exit = 2, "The machine was not touched"), not taken. QEMU cannot stand in, because slirp never scales. Post the reading at the head being landed, with command, exit and log. It must show rcv_shift 7 negotiated with the CDN, and mbps and rtt_ms read together, so that window growth past 65,535 can be told apart from a plateau. The body's own close test is mbps > 524.3 / min(rtt_ms).

• toyos-net-shard/tcp/src/rx.rs:393-395 — the rate the buffer grows at is a claim this change makes ("twice what the user read per round trip"), and no test can fail on it. The throughput test only asserts the window reaches 4 MiB, and a_receive_buffer_nobody_reads_never_grows only covers zero reads followed by unlimited reads. I expect both of these patches to pass every test:

  • (a) let want = usize::try_from(per_rtt.saturating_mul(2)) → saturating_mul(64)
  • (b) rtt.filter(|&rtt| now.since(began) >= rtt) → rtt, so the buffer is measured and grown on every read.

  Either makes one small read grow a connection toward the full 4 MiB. The implementer runs both and adds the host test that turns them red. That test reads at a fixed rate R for several round trips and asserts the window stays at or below max(65,535, 2·R·SRTT + one MSS).

• toyos-net-shard/tcp/src/rx.rs:208 — nothing checks the round-up's unit <= self.free() guard, and that guard alone holds the module invariant unread + (edge − next) ≤ capacity. No property in props.rs asserts that invariant (check asserts edge order, bounds and pipe, never capacity). The case is reachable only at shift > 0, with a hole open and fewer than one unit free. Run the mutation && unit <= self.free() → (deleted). If it survives, add a tests/receive.rs case at shift 7: a hole open, under 128 bytes free and a window under 128 bytes, asserting the field stays 0 and the edge holds. Also add the invariant to props.rs's check.

NOTE

• toyos-net-shard/tcp/src/conn.rs:711 / rx.rs:390 — growth reads the sender-side SRTT, which sample updates only from ACKs of this end's data. A pure downloader's SRTT therefore stays at the handshake's value. Under queueing, once the real RTT exceeds twice that value, want = 2·read·S/elapsed drops below the current window and growth stops. Linux's tcp_rcv_space_adjust uses the receiver's own RTT estimate (tcp_rcv_rtt_measure_ts), which this connection could take from TSecr. The T14 reading will show whether this caps the window. If it does, that is this rule's defect, not the link's.
• Memory bound: held, by reading. places_for gives total_mem / 8 / PLACE_BYTES places. held() counts streams, listeners, sockets and tcp_orphans. A connect, listen, accept or bind with no place left is refused with nothing made. Per place, the rings are capped at the configured 4 MiB, and an unaccepted child stays at 65,535 because only a read grows it. So no peer makes netstack allocate past places × 12 MiB.
• PR body and commit message say "a stalled reader … hold at most 65,535". That is false at netstack. sockets.bridge drains a connection into its client's 2 MiB pipe whether or not the client reads, so a client that never reads still grows its connection's receive buffer toward 4 MiB. It stays inside PLACE_BYTES. The claim is true only of the toyos-net-tcp user.
• PR body, "Independent oracle" — the claim that nothing below the T14 can be the peer does not hold. smoltcp is a general, widely used, independent Rust TCP that implements RFC 7323 window scaling. As a dev-dependency peer inside the host test network it would be a differential implementation for negotiation, scaled edges and the round-up, with no TAP device and no privileges. Its absence leaves this ours-against-ours below the T14.
• The gate logs (ci-host.log, guest.log) do not record the head they ran at. Only their timestamps and the body tie them to d286ece79, and the worktree now sits at the metal merge 751dfc6f6.

SEND BACK

[Japabu]

Mutation patches of round 2, at cd34232c9. Each applied with git apply after git apply --check, then cargo test --offline --no-fail-fast in toyos-net-shard/tcp, then git apply -R; every one EXIT=101, RESTORED=0, tree clean after. Which tests each reds is in the body.

g1-growth-x64

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 76e6b3979..dd076a25b 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -442,7 +442,7 @@ impl Rx {
         if let Some(rtt) = self.rtt.filter(|&rtt| now.since(began) >= rtt) {
             let read = u128::try_from(read).unwrap_or(u128::MAX);
             let per_rtt = read.saturating_mul(rtt.as_nanos()).checked_div(now.since(began).as_nanos()).unwrap_or(0);
-            let want = usize::try_from(per_rtt.saturating_mul(2)).unwrap_or(usize::MAX);
+            let want = usize::try_from(per_rtt.saturating_mul(64)).unwrap_or(usize::MAX);
             self.buf.grow(want.min(self.max));
             self.round = (now, 0);
         }

g2-grown-on-every-read

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 76e6b3979..9c3d39c92 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -439,7 +439,7 @@ impl Rx {
         let (began, read) = self.round;
         let read = read.saturating_add(n);
         self.round = (began, read);
-        if let Some(rtt) = self.rtt.filter(|&rtt| now.since(began) >= rtt) {
+        if let Some(rtt) = self.rtt {
             let read = u128::try_from(read).unwrap_or(u128::MAX);
             let per_rtt = read.saturating_mul(rtt.as_nanos()).checked_div(now.since(began).as_nanos()).unwrap_or(0);
             let want = usize::try_from(per_rtt.saturating_mul(2)).unwrap_or(usize::MAX);

g3-no-echo-sample

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 9bbe2dec2..122194be1 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -145,7 +145,7 @@ impl Rx {
             Sampler::Echo(last) => {
                 let Some((echo, Some(age))) = echo.filter(|&(echo, _)| len >= mss && *last != Some(echo)) else { return };
                 *last = Some(echo);
-                self.rtt = Some(self.rtt.map_or(age, |rtt| rtt.saturating_mul(7).saturating_add(age).checked_div(8).unwrap_or(age)));
+                let _ = age;
             }
             Sampler::Fill(mark) => {
                 if let Some((edge, since)) = *mark {

g4-fill-keeps-the-largest

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 76e6b3979..9df0dfbf5 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -153,7 +153,7 @@ impl Rx {
                         return;
                     }
                     let took = now.since(since).max(Duration::from_micros(1));
-                    self.rtt = Some(self.rtt.map_or(took, |rtt| rtt.min(took)));
+                    self.rtt = Some(self.rtt.map_or(took, |rtt| rtt.max(took)));
                 }
                 *mark = Some((self.edge, now));
             }

p1-no-loss-probe

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index 7de89bf2e..9d12ce79d 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -739,6 +739,9 @@ impl Sync {
     /// RTO, which it then stands in for.
     fn schedule_probe(&mut self, now: Instant) {
         self.probe_at = None;
+        if self.probe_at.is_none() {
+            return;
+        }
         let Some(srtt) = self.rtt.srtt() else { return };
         let Some(rto) = self.rtx_timer else { return };
         if !self.sack_ok || self.recovery != Recovery::None || !self.tx.sacked().is_empty() || self.probe.is_some() || self.persist.is_some() {

p2-no-loss-response

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index 7de89bf2e..69fb6ef70 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -643,7 +643,7 @@ impl Sync {
         // RFC 8985 §7.4: a retransmitted probe acknowledged without a D-SACK repaired a loss.
         if let Some((_, resent)) = self.probe.filter(|&(end, _)| ack.at_or_after(end) && acked <= flight) {
             self.probe = None;
-            if resent && !dsack {
+            if resent && !dsack && false {
                 self.cc.on_loss(flight);
                 self.cc.cwnd = self.cc.cwnd.min(self.cc.ssthresh);
                 self.cc.end_recovery();

p3-dsack-ignored

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index 7de89bf2e..b8cb1df52 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -643,7 +643,7 @@ impl Sync {
         // RFC 8985 §7.4: a retransmitted probe acknowledged without a D-SACK repaired a loss.
         if let Some((_, resent)) = self.probe.filter(|&(end, _)| ack.at_or_after(end) && acked <= flight) {
             self.probe = None;
-            if resent && !dsack {
+            if resent {
                 self.cc.on_loss(flight);
                 self.cc.cwnd = self.cc.cwnd.min(self.cc.ssthresh);
                 self.cc.end_recovery();

p4-no-delayed-ack-allowance

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index 7de89bf2e..c529e08d0 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -744,7 +744,7 @@ impl Sync {
         if !self.sack_ok || self.recovery != Recovery::None || !self.tx.sacked().is_empty() || self.probe.is_some() || self.persist.is_some() {
             return;
         }
-        let delayed = if self.tx.flight() <= self.smss() { WORST_DELAYED_ACK } else { PROBE_SLACK };
+        let delayed = if self.tx.flight() <= self.smss() { PROBE_SLACK } else { PROBE_SLACK };
         self.probe_at = Some(now.after(srtt.saturating_mul(2).saturating_add(delayed)).min(rto));
     }
 

r1-round-up-unguarded

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 76e6b3979..316c6ef97 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -325,7 +325,7 @@ impl Rx {
         // room, as Linux does: else a sender owing the text of a hole waits on a window not shut.
         let unit = 1u32.checked_shl(u32::from(self.shift)).unwrap_or(u32::MAX);
         let window = edge.since(self.next);
-        let edge = if window > 0 && window < unit && unit <= self.free() { self.next.add(unit) } else { edge };
+        let edge = if window > 0 && window < unit { self.next.add(unit) } else { edge };
         let field = (edge.since(self.next) >> self.shift).min(u32::from(u16::MAX));
         (edge, u16::try_from(field).unwrap_or(u16::MAX))
     }

[Japabu]

Review of #820, round 2, at cd34232c9. This review covers the round's own change, 35b407984..cd34232c9.

Net lines. Whole branch against origin/main: 13 files, +534/−66. Production (toyos-net-shard/tcp/src without the cfg(test) props.rs, plus netstack): about +255/−44. Tests: +271/−21 (props.rs included). This round alone: production src +166/−14, tests +162/−21.

Round 1's BLOCKERs

• s_net_005 65,535 arm: CLOSED. The arm is restored at both parities beside 4 MiB. tcp-tests.log at cd34232c9 gives EXIT=0. mut-p1-no-loss-probe.log reds s_net_005_lost_acks at net.rs:21 (the transfer did not finish), which is the base's stall.
• T14 reading at the head being landed: OPEN. metal-stage.exit is EXIT=2 at 0350817cd, and its log says "The machine was not touched, so this run establishes nothing about it". No reading at this head is posted on the PR.
• Growth rate has no failing test: CLOSED. mut-g1-growth-x64.log and mut-g2-grown-on-every-read.log both red the_receive_window_grows_by_what_is_read_per_round_trip (net.rs:469).
• Round-up guard: CLOSED. mut-r1-round-up-unguarded.log reds a_sub_unit_window_rounds_up_only_into_free_room and 12 property runs at the new props.rs:139 invariant.

BLOCKER

• toyos-net-shard/tcp/src/conn.rs:644 — RFC 8985 §7.4 is not followed. The PR infers a repaired loss when the ACK equals TLP.end_seq. RFC 8985 §7.4.2's TLP_process_ack reacts only when the ACK number is greater than TLP.end_seq (Linux: after(ack, tlp_high_seq)). An ACK that equals it with no D-SACK leaves the episode open. With at_or_after, a delayed ACK for the original last segment that arrives after the probe left halves cwnd for a loss that never happened. The D-SACK that follows finds probe already None. tests/recovery.rs:364 (rfc_8985_7_4_a_resent_probe_acknowledged_without_a_dsack_repaired_a_loss, which ACKs exactly 15,481 = end) asserts this deviation under the RFC's name. Fix: infer loss only for ack.after(end), and keep the probe outstanding at ack == end without a D-SACK. Then add a test: the ACK equals end with no D-SACK, then the D-SACK arrives, and assert tcp.loss-probe-recovery stays 0 and cwnd stands. Move the loss case to an ACK past end.

• toyos-net-shard/tcp/src/conn.rs:1354 — RFC 8985 §7.3's second MUST is missing: no probe unless an RTT sample was taken since the last probe. On a SACK connection without timestamps, hand_off clears timing on every retransmission (conn.rs:1160). Once the path's RTT rises past 2·SRTT, the probe (a resend of the tail) always leaves before the timed segment's ACK, so no sample is ever taken. SRTT stays where it was, and every tail gets a spurious probe for the rest of the connection. That condition exists to stop exactly this. Add the condition. Add a test: no timestamps, SACK on, the path's RTT raised after the handshake (as the_receive_window_grows… does), a sender writing a tail per round for several rounds. Assert tcp.loss-probe ≤ 1 and that SRTT reaches the new RTT.

• toyos-net-shard/tcp/src/conn.rs:740-749 and :782-822 — two RFC 8985 exclusions are missing.

  • §7.2: no probe is scheduled in RTO recovery. After expire, recovery is None and the scoreboard is cleared, so the first cumulative ACK without SACK schedules a probe while go-back-N (rtx_next) is still running. A probe then goes outside a one-segment cwnd. If that probe is a resend acknowledged without a D-SACK, it cuts ssthresh a second time for the same episode.
  • §7.1: TLP state is reset on entering fast recovery. duplicate clears probe_at but not probe, so a resent probe still outstanding when SACK recovery begins applies on_loss again when it is acknowledged.

  Fix both: guard on RTO recovery, and reset probe on entering recovery. Add a test: s_lr_019's timeline (probe at 22 ms, RTO at 222 ms), then a plain ack(2449) without SACK at 232 ms, then time advanced past 2·SRTT + 2 ms. Assert no tcp.loss-probe beyond the first before the RTO. I expect this to fail today.

• toyos-net-shard/tcp/src/rx.rs:146 — "full-sized" is measured against this end's send SMSS (cc.smss, conn.rs:551), not against what the peer sends. Linux's tcp_rcv_rtt_measure_ts uses icsk_ack.rcv_mss, which is learned from received segments. A timestamped peer whose segments are smaller than our SMSS never yields a sample: a peer behind a PMTU-reduced tunnel without MSS clamping, or one that sizes its own segments. For such a peer rcv_rtt stays None, the buffer stays at 65,535 for the life of the connection, and nothing records it. Because the Echo sampler is the only sampler on timestamped connections, nothing falls back. Fix: measure against the received segment size, as Linux does. Add a test: shift 7 with timestamps, the peer sends 1,380-byte segments, and the reader keeps up. Assert rcv_rtt is Some and the capacity grows past 65,535.

• toyos-net-shard/tcp/src/stack.rs:71-73, rx.rs:132,137 — Info.rcv_capacity, Info.rcv_rtt, Rx::rtt() and Rx::capacity() have no shipping reader. Tcp::info is called by no code in userland/ or toyos-net-shard/src, and props.rs is cfg(test). The only non-test reader named in the body is the measurement commit, which never lands. That is code shipped for tests alone. Either assert through what the tests already observe, or give these fields a shipping reader that lands.

NOTE

• rx.rs:146-148 — an echo of the last ACK sent before an idle period ages by the length of that idle. One such sample moves the estimate by idle/8, and after_read then waits that long before the next growth decision. A peer can do the same on purpose by echoing an old TSval at or after first, though it only starves its own connection. Linux's tcp_rtt_tsopt_us also floors a zero-tick delta at one tick, whereas a 0 ms age here is taken as a sample.
• conn.rs:1083 — after a PTO fires (rtx_timer cleared, probe_due set), an ACK that opens persist leaves probe_due set. The next next_segment then resends the last segment into a zero window instead of starting persist alone.
• s_lr_018 / s_lr_019: the edit is honest. Both tests keep every assertion, now 22 ms later behind an asserted probe, and the RTO and SACK relationship they test is unchanged (the RTO now runs from 22 ms, so 222 ms, then ACKs at 232–234 ms instead of 200, then 210–212). fixture_e still covers the RTO with no probe. No test covers a SACK connection's first RTO at its original time, but RFC 8985 §7.2 makes the probe stand in for that.
• PR body, "Tests and their controls": rfc_8985_7_4_…repaired_a_loss is described as RFC 8985 §7.4's case. At ack == end it is not (see the first BLOCKER).

SEND BACK

[Japabu]

T14 at 0350817cd (wt/toyos-wscale-metal: this branch's cd34232c9 merged with #818's row plus the measurement-only commit), run by the orchestrator: download boot, image sha256 75ee3357…f0204d9a checked against request.txt, toyos-metal --fat32-check exit 0; judge EXIT=0. A regression: [download] cut bytes=120113056 secs=39.810 mbps=24.1 rtt_ms=[17.4 15.3 15.0 14.8 15.0] busy=0.012 cpu_s=3.833 cpu_ms_per_mb=31.91: the first download was cut at the window after 120 MB, so no second (warm) run happened. Round 1's head (d286ece79, reading above) did 328.2 Mb/s on the same machine and URL; Ubuntu does 750–828 warm and 330 cold. Something this round added (the tail-loss probe, the receiver-side RTT for growth, or the measurement commit itself) cuts throughput by about 13x. The [tcp] lines the measurement commit was to print do not appear in the readback's judge output.

[Japabu]

Mutation patches of round 3, at 6df8222c1. Each applied with git apply after git apply --check to a copy of toyos-net-shard/tcp and toyos-net-wire at that head, then cargo test --offline --no-fail-fast in toyos-net-shard/tcp, then git apply -R; every one EXIT=101, RESTORED=0, the copy clean after (git status --porcelain empty). The base run on the unmutated copy: EXIT=0, 405 passed. Which tests each reds is in the body.

a1-probe-loss-at-its-end

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..1508dc4 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -649,7 +649,7 @@ impl Sync {
         if let Some((end, resent)) = self.probe.filter(|&(end, _)| ack.at_or_after(end) && acked <= flight) {
             if !resent || dsack == Some(end) || (same && seg.options.sack_blocks().len() == 0) {
                 self.probe = None;
-            } else if ack.after(end) {
+            } else if ack.at_or_after(end) {
                 self.probe = None;
                 self.cc.on_loss(flight);
                 self.cc.cwnd = self.cc.cwnd.min(self.cc.ssthresh);

a2-any-dsack-ends-the-episode

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..ecfd397 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -647,7 +647,7 @@ impl Sync {
         // a duplicate without SACK ends the episode with nothing lost; only an ACK past the end
         // without either says a resent probe repaired a loss.
         if let Some((end, resent)) = self.probe.filter(|&(end, _)| ack.at_or_after(end) && acked <= flight) {
-            if !resent || dsack == Some(end) || (same && seg.options.sack_blocks().len() == 0) {
+            if !resent || dsack.is_some() || (same && seg.options.sack_blocks().len() == 0) {
                 self.probe = None;
             } else if ack.after(end) {
                 self.probe = None;

a3-no-duplicate-case

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..9f7e403 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -647,7 +647,7 @@ impl Sync {
         // a duplicate without SACK ends the episode with nothing lost; only an ACK past the end
         // without either says a resent probe repaired a loss.
         if let Some((end, resent)) = self.probe.filter(|&(end, _)| ack.at_or_after(end) && acked <= flight) {
-            if !resent || dsack == Some(end) || (same && seg.options.sack_blocks().len() == 0) {
+            if !resent || dsack == Some(end) {
                 self.probe = None;
             } else if ack.after(end) {
                 self.probe = None;

e1-echo-unclamped

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 8e98cde..e1999d9 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -160,7 +160,7 @@ impl Rx {
                 *last = Some(echo);
                 let Some(age) = age.filter(|_| len >= self.rcv_mss) else { return };
                 let age = age.max(TICK);
-                self.rtt = Some(self.rtt.map_or(age, |rtt| rtt.saturating_mul(7).saturating_add(age.min(rtt.saturating_mul(2))).checked_div(8).unwrap_or(rtt)));
+                self.rtt = Some(self.rtt.map_or(age, |rtt| rtt.saturating_mul(7).saturating_add(age).checked_div(8).unwrap_or(rtt)));
             }
             Sampler::Fill(mark) => {
                 if let Some((edge, since)) = *mark {

e2-zero-tick-taken

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 8e98cde..ed5c430 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -159,7 +159,7 @@ impl Rx {
                 let Some((echo, age)) = echo.filter(|&(echo, _)| *last != Some(echo)) else { return };
                 *last = Some(echo);
                 let Some(age) = age.filter(|_| len >= self.rcv_mss) else { return };
-                let age = age.max(TICK);
+                let age = age.max(Duration::ZERO);
                 self.rtt = Some(self.rtt.map_or(age, |rtt| rtt.saturating_mul(7).saturating_add(age.min(rtt.saturating_mul(2))).checked_div(8).unwrap_or(rtt)));
             }
             Sampler::Fill(mark) => {

g1-growth-x64

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 8e98cde..0511578 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -472,7 +472,7 @@ impl Rx {
         if let Some(rtt) = self.rtt.filter(|&rtt| now.since(began) >= rtt) {
             let read = u128::try_from(read).unwrap_or(u128::MAX);
             let per_rtt = read.saturating_mul(rtt.as_nanos()).checked_div(now.since(began).as_nanos()).unwrap_or(0);
-            let want = usize::try_from(per_rtt.saturating_mul(2)).unwrap_or(usize::MAX);
+            let want = usize::try_from(per_rtt.saturating_mul(64)).unwrap_or(usize::MAX);
             self.buf.grow(want.min(self.max));
             self.round = (now, 0);
         }

g2-grown-on-every-read

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 8e98cde..2c4bcaa 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -469,7 +469,7 @@ impl Rx {
         let (began, read) = self.round;
         let read = read.saturating_add(n);
         self.round = (began, read);
-        if let Some(rtt) = self.rtt.filter(|&rtt| now.since(began) >= rtt) {
+        if let Some(rtt) = self.rtt {
             let read = u128::try_from(read).unwrap_or(u128::MAX);
             let per_rtt = read.saturating_mul(rtt.as_nanos()).checked_div(now.since(began).as_nanos()).unwrap_or(0);
             let want = usize::try_from(per_rtt.saturating_mul(2)).unwrap_or(usize::MAX);

g3-no-echo-sample

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 8e98cde..3d455be 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -160,7 +160,7 @@ impl Rx {
                 *last = Some(echo);
                 let Some(age) = age.filter(|_| len >= self.rcv_mss) else { return };
                 let age = age.max(TICK);
-                self.rtt = Some(self.rtt.map_or(age, |rtt| rtt.saturating_mul(7).saturating_add(age.min(rtt.saturating_mul(2))).checked_div(8).unwrap_or(rtt)));
+                let _ = age;
             }
             Sampler::Fill(mark) => {
                 if let Some((edge, since)) = *mark {

g4-fill-keeps-the-largest

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 8e98cde..210c521 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -168,7 +168,7 @@ impl Rx {
                         return;
                     }
                     let took = now.since(since).max(Duration::from_micros(1));
-                    self.rtt = Some(self.rtt.map_or(took, |rtt| rtt.min(took)));
+                    self.rtt = Some(self.rtt.map_or(took, |rtt| rtt.max(took)));
                 }
                 *mark = Some((self.edge, now));
             }

m1-full-sized-is-our-mss

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 8e98cde..fac8b7f 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -158,7 +158,7 @@ impl Rx {
             Sampler::Echo(last) => {
                 let Some((echo, age)) = echo.filter(|&(echo, _)| *last != Some(echo)) else { return };
                 *last = Some(echo);
-                let Some(age) = age.filter(|_| len >= self.rcv_mss) else { return };
+                let Some(age) = age.filter(|_| len >= self.mss_bounds.1) else { return };
                 let age = age.max(TICK);
                 self.rtt = Some(self.rtt.map_or(age, |rtt| rtt.saturating_mul(7).saturating_add(age.min(rtt.saturating_mul(2))).checked_div(8).unwrap_or(rtt)));
             }

m2-full-sized-never-lowered

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 8e98cde..c7d56a4 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -184,9 +184,7 @@ impl Rx {
             self.rcv_mss = len.min(offered);
         } else if len >= floor {
             self.short = len;
-            if len == short {
-                self.rcv_mss = len;
-            }
+            let _ = short;
         }
     }
 

p1-no-loss-probe

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..765065a 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -746,6 +746,9 @@ impl Sync {
     /// slack when more are, and never after the RTO, which it then stands in for.
     fn schedule_probe(&mut self, now: Instant) {
         self.probe_at = None;
+        if self.probe_at.is_none() {
+            return;
+        }
         let Some(srtt) = self.rtt.srtt() else { return };
         let Some(rto) = self.rtx_timer else { return };
         let recovering = self.recovery != Recovery::None || (self.episode && self.tx.una.at_or_before(self.recover));

p2-no-loss-response

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..3922098 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -649,7 +649,7 @@ impl Sync {
         if let Some((end, resent)) = self.probe.filter(|&(end, _)| ack.at_or_after(end) && acked <= flight) {
             if !resent || dsack == Some(end) || (same && seg.options.sack_blocks().len() == 0) {
                 self.probe = None;
-            } else if ack.after(end) {
+            } else if ack.after(end) && false {
                 self.probe = None;
                 self.cc.on_loss(flight);
                 self.cc.cwnd = self.cc.cwnd.min(self.cc.ssthresh);

p3-dsack-ignored

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..158a1a8 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -647,7 +647,7 @@ impl Sync {
         // a duplicate without SACK ends the episode with nothing lost; only an ACK past the end
         // without either says a resent probe repaired a loss.
         if let Some((end, resent)) = self.probe.filter(|&(end, _)| ack.at_or_after(end) && acked <= flight) {
-            if !resent || dsack == Some(end) || (same && seg.options.sack_blocks().len() == 0) {
+            if !resent || false || (same && seg.options.sack_blocks().len() == 0) {
                 self.probe = None;
             } else if ack.after(end) {
                 self.probe = None;

p4-no-delayed-ack-allowance

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..43f94da 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -752,7 +752,7 @@ impl Sync {
         if !self.sack_ok || recovering || !self.tx.sacked().is_empty() || self.probe.is_some() || !self.sampled || self.persist.is_some() {
             return;
         }
-        let delayed = if self.tx.flight() <= self.smss() { WORST_DELAYED_ACK } else { PROBE_SLACK };
+        let delayed = if self.tx.flight() <= self.smss() { PROBE_SLACK } else { PROBE_SLACK };
         self.probe_at = Some(now.after(srtt.saturating_mul(2).saturating_add(delayed)).min(rto));
     }
 

r1-round-up-unguarded

diff --git a/toyos-net-shard/tcp/src/rx.rs b/toyos-net-shard/tcp/src/rx.rs
index 8e98cde..21c4577 100644
--- a/toyos-net-shard/tcp/src/rx.rs
+++ b/toyos-net-shard/tcp/src/rx.rs
@@ -355,7 +355,7 @@ impl Rx {
         // room, as Linux does: else a sender owing the text of a hole waits on a window not shut.
         let unit = 1u32.checked_shl(u32::from(self.shift)).unwrap_or(u32::MAX);
         let window = edge.since(self.next);
-        let edge = if window > 0 && window < unit && unit <= self.free() { self.next.add(unit) } else { edge };
+        let edge = if window > 0 && window < unit { self.next.add(unit) } else { edge };
         let field = (edge.since(self.next) >> self.shift).min(u32::from(u16::MAX));
         (edge, u16::try_from(field).unwrap_or(u16::MAX))
     }

s1-no-sample-condition

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..b7ee310 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -749,7 +749,7 @@ impl Sync {
         let Some(srtt) = self.rtt.srtt() else { return };
         let Some(rto) = self.rtx_timer else { return };
         let recovering = self.recovery != Recovery::None || (self.episode && self.tx.una.at_or_before(self.recover));
-        if !self.sack_ok || recovering || !self.tx.sacked().is_empty() || self.probe.is_some() || !self.sampled || self.persist.is_some() {
+        if !self.sack_ok || recovering || !self.tx.sacked().is_empty() || self.probe.is_some() || self.persist.is_some() {
             return;
         }
         let delayed = if self.tx.flight() <= self.smss() { WORST_DELAYED_ACK } else { PROBE_SLACK };

s2-probe-in-rto-recovery

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..4a66310 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -748,7 +748,7 @@ impl Sync {
         self.probe_at = None;
         let Some(srtt) = self.rtt.srtt() else { return };
         let Some(rto) = self.rtx_timer else { return };
-        let recovering = self.recovery != Recovery::None || (self.episode && self.tx.una.at_or_before(self.recover));
+        let recovering = self.recovery != Recovery::None;
         if !self.sack_ok || recovering || !self.tx.sacked().is_empty() || self.probe.is_some() || !self.sampled || self.persist.is_some() {
             return;
         }

s3-probe-kept-into-fast-recovery

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..e37d459 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -815,8 +815,7 @@ impl Sync {
             return;
         }
         // RFC 8985 §7.1: fast recovery starts the loss probe's state afresh.
-        (self.probe_at, self.probe_due, self.probe) = (None, false, None);
-        self.arm(ctx.now);
+        self.probe_at = None;
         let flight = self.tx.flight();
         self.cc.on_loss(flight.saturating_sub(self.lt_bytes));
         self.recover = self.tx.nxt.sub(1);

s4-due-probe-kept-into-persist

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac..f030145 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -846,7 +846,7 @@ impl Sync {
             if self.persist.is_none() {
                 let rto = self.rtt.rto();
                 self.persist = Some(Persist { at: now.after(rto), interval: rto, from: self.tx.nxt, due: false, unanswered: false });
-                (self.rtx_timer, self.probe_at, self.probe_due) = (None, None, false);
+                (self.rtx_timer, self.probe_at) = (None, None);
             }
         } else if let Some(persist) = self.persist.take() {
             if self.tx.nxt.after(persist.from) {

[Japabu]

Review of #820, round 3, at 6df8222c1. This round's own change is ba9ea6be9..6df8222c1.

Net lines. Whole branch against origin/main: 13 files, +806/−68. Production (toyos-net-shard/tcp/src without props.rs and rx.rs's 65-line test module, plus netstack): +295/−44. Tests: +511/−24. This round: 7 files, +328/−58. It deletes Info.rcv_capacity, Info.rcv_rtt and Rx::rtt().

Earlier BLOCKERs

• Round 1, T14 reading at the head being landed: OPEN. metal-stage.log at 30c6678ec exits 2 and says "The machine was not touched, so this run establishes nothing about it". The only hardware reading posted is at 0350817cd (round 2): 24.1 Mb/s, a regression. The cause and the fix are shown only in the host network (repro-r2-cd34232c9.log: 34.2 and 34.3 Mb/s at 1,428- and 1,380-byte segments; repro-r3-6df8222c1.log: 1202.4 Mb/s for both, window 4 MiB). The CDN's segment size on the T14's path is inferred, not read. A reading that contains 6df8222c1 unchanged is still owed. It must give rcv_shift 7, rcv_mss, and mbps and rtt_ms read together.
• Round 2, §7.4.2 loss inferred at ack == end: CLOSED. mut/a1-probe-loss-at-its-end.log exits 101 and reds the four rfc_8985_7_4_… tests at recovery.rs:367/387/479/494. The branch at conn.rs:649-658 has the same order as Linux's tcp_process_tlp_ack. dsack == Some(end) is FLAG_DSACK_TLP. a2 and a3 red their own tests.
• Round 2, §7.3 RTT-sample condition: CLOSED. mut/s1-no-sample-condition.log exits 101 (net.rs:537). Judged against RFC 8985 §7.3: condition 2 forbids a probe only until an RTT measurement has been taken since the last probe. It does not forbid one in each round that follows a sample. With one Karn-timed sample a round and gain 1/8, SRTT goes from 20 ms to 30, 38.75, 46.4 and 53.1 ms. Only the fourth value passes (100 − 2)/2, so probes in rounds 1, 3, 5 and 7 are what the RFC allows. My round-2 request for "at most one" was stricter than the RFC. The test asserts the RFC's bound: no two rounds in a row probe, none after round 20, and SRTT reaches the path's.
• Round 2, §7.2 and §7.1 exclusions: CLOSED. mut/s2-probe-in-rto-recovery.log reds rfc_8985_7_2_no_probe_in_rto_recovery (recovery.rs:429). mut/s3-probe-kept-into-fast-recovery.log reds rfc_8985_7_1_fast_recovery_ends_the_probes_episode (recovery.rs:448).
• Round 2, full-sized measured against the send MSS: CLOSED. mut/m1-full-sized-is-our-mss.log reds rx::tests::full_sized_is_learned_from_what_arrives (rx.rs:511) and a_peer_sending_segments_shorter_than_our_send_mss_still_grows_the_window (net.rs:498). m2 reds the lowering rule. measure_mss follows tcp_measure_rcv_mss. The upper bound path_mss − TS_OPTION is what stack.rs:415 puts in the SYN.
• Round 2, Info/Rx getters shipped for tests: CLOSED. rcv_capacity, rcv_rtt and Rx::rtt() are deleted. Rx::capacity() is #[cfg(test)].

Round 2's NOTEs are answered. The idle-aged echo is fixed (e1 and e2 red an_echo_aged_by_the_peers_silence…). The due probe that went into persist is fixed (s4 reds a_probe_due_when_the_window_shuts…).

Gates at 6df8222c1: crate tests EXIT=0; cargo run -- --ci host EXIT=0, "78 step(s), all green"; --build-only EXIT=0; guest suite EXIT=0. The negative control (round 2's src under this round's tests) exits 101 and names --test net and --test recovery. Every log's first line names the head. No guest test is added or changed.

BLOCKER

• PR body, "Gates", T14: round 1's hardware BLOCKER above, still open. This round exists because the last hardware reading regressed 13x. The host reproduction shows the mechanism, not the T14 path.

NOTE

• toyos-net-shard/tcp/tests/recovery.rs:1: the module doc's first line is broken by a stray paste: //! L expect(&h.send(32, 1448), &["SEQ=15481 LEN=1448"]);ss recovery: NewReno on E, …. On origin/main it reads //! Loss recovery: …. Restore the line.
• toyos-net-shard/tcp/src/conn.rs:652-663: if one ACK goes past a resent probe's end and also carries new SACK information that enters recovery, cwnd is cut twice in one window. §7.4.2's on_loss runs first, then duplicate runs on_loss again. Linux's tcp_enter_recovery skips tcp_init_cwnd_reduction when the state is already CWR. The effect is conservative throughput, not a safety problem, and no test reaches it. Either skip the TLP reduction when the same ACK enters recovery, or record the gap.

SEND BACK

[Japabu]

T14 at 30c6678ec (wt/toyos-wscale-metal: this branch's 6df8222c1 merged with #818's row plus the measurement-only commit), run by the orchestrator: download boot, image sha256 e2c09588…36b8a6fb checked against request.txt, toyos-metal --fat32-check exit 0; judge EXIT=0, PASS internet_download. Two downloads back to back:

• first: mbps=107.1 secs=12.728 cpu_ms_per_mb=15.40; [tcp] rcv_shift=7 rcv_capacity=4194304 rcv_rtt=Some(48.4ms) rcv_mss=1424 srtt=15.46ms loss_probe=0 retransmit_bytes=0
• second: mbps=662.4 secs=2.058 cpu_ms_per_mb=13.52 busy=0.140 (busiest CPUs 0.688 and 0.383); [tcp] rcv_shift=7 rcv_capacity=4194304 rcv_rtt=Some(12.4ms) rcv_mss=1424
• the five short connections (RTT probes) end at rcv_capacity=65535 rcv_rtt=None rcv_mss=524, as expected for connections that read nothing.
  rtt_ms=[17.8 15.7 15.0 16.0 16.1]. Ubuntu on the same machine, cable and URL, measured by the orchestrator immediately after (curl, project User-Agent): 390.3 Mb/s cold, 848.3 and 852.5 warm; TCP connect 15.5–35.4 ms, TLS ready 42–67 ms; whole-system CPU 0.26 s per warm download (about 1.5 ms per MB), 0.53 s cold. The window now scales to 4 MiB on hardware (rcv_shift=7, rcv_mss=1424): round 2's regression is gone, and the 64 KiB issue's exit is met. Remaining gaps against Ubuntu: warm 662 vs 850 Mb/s with one CPU at 69%; cold 107 vs 390 (the first connection's window grew on a 48 ms receiver RTT); CPU about 9x per MB.

[Japabu]

Mutation patch of round 4, at a700899ee. Applied with git apply after git apply --check in the worktree, then cargo test --offline --no-fail-fast in toyos-net-shard/tcp: EXIT=101, only rfc_8985_7_4_an_ack_that_infers_the_probes_loss_and_enters_recovery_cuts_once red (left (4054, 4054), right (5068, 5068)); git apply -R RESTORED=0, git status --porcelain empty after. The unmutated tree: EXIT=0, 406 passed.

c1-recovery-cuts-again

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -822,3 +822,3 @@
         // One ACK is one congestion event, cut once, as Linux's `tcp_enter_recovery` in CWR.
-        if !cut {
+        if !cut || true {
             self.cc.on_loss(flight.saturating_sub(self.lt_bytes));

[Japabu]

Review of #820, round 4, at a700899ee. This round's own change is 6df8222c1..a700899ee.

Net lines. Whole branch against origin/main: 13 files, +835/−73. This round: 2 files, +31/−7. Production is conn.rs, +12/−6. Tests are recovery.rs, +19/−1.

Earlier BLOCKERs

• Round 1, T14 reading at the head being landed: CLOSED. The reading was taken at 30c6678ec. git merge-base --is-ancestor 6df8222c1 30c6678ec holds, and the image's sha256 e2c09588…36b8a6fb matches request.txt. Judge EXIT=0, PASS internet_download. Both downloads give rcv_shift=7 rcv_capacity=4194304 rcv_mss=1424, at 107.1 and 662.4 Mb/s, with rtt_ms=[17.8 15.7 15.0 16.0 16.1] read alongside. Round 2's 24.1 Mb/s is gone. The reading also stands for a700899ee. 6df8222c1..a700899ee touches only conn.rs and recovery.rs. The new code acts only when self.probe holds a resent probe whose loss is inferred. Every tcp closed line in that boot's kernel.log (361–371) shows the stack-wide counters at loss_probe=0 loss_probe_recovery=0 sack_recovery=0 retransmit_bytes=0. So cut was false on every ACK of that run, and the new head behaves the same on that path. This also holds if the change below is deleted.

Round 3's NOTEs:

• recovery.rs:1 doc line: CLOSED. It reads //! Loss recovery: … again.
• Double cut on one ACK: answered with new code, but my round-3 premise was wrong. See the first BLOCKER.

BLOCKER

• toyos-net-shard/tcp/src/conn.rs:649,659,795,825: the cut flag rests on a false oracle. That oracle was mine, in round 3. Linux v6.12 cuts twice on this ACK.

  • tcp_ack runs tcp_process_tlp_ack (tcp_input.c:4038) before tcp_fastretrans_alert (:4048).
  • The loss branch of tcp_process_tlp_ack (:3846-3849) runs tcp_init_cwnd_reduction, sets CWR and runs tcp_end_cwnd_reduction. It then runs tcp_try_keep_open, which moves the state to Open, or to Disorder when anything is SACKed (:2765-2770).
  • So when tcp_enter_recovery (:2915-2919) checks tcp_in_cwnd_reduction (CWR or Recovery only, tcp.h:1333), the check is false. It calls tcp_init_cwnd_reduction again: a second ssthresh(), and a second W_max update in CUBIC.

  The code before this round matched Linux. This round adds a parameter, a flag and a branch whose only stated reason is false. The other reason would be RFC 5681/6675's one reduction per window. That reason does not fit this code either: the probe's cut sets no recover, so an ACK one later in the same window that enters SACK recovery still cuts a second time. The guard therefore covers one ACK, not one congestion event.

  • Named change: delete cut and restore duplicate(ack, ctx), which is net −6 production lines. Keep the test, but assert Linux's sequence: (4054, 4054), the second cut from the flight after the ACK. Cite tcp_process_tlp_ack → tcp_try_keep_open → tcp_enter_recovery as its oracle.
  • The alternative, only with a reason I can check: make the probe's reduction the episode's. It must set a recovery point that stops later entries in that window from cutting again, without stopping them from retransmitting. Add a test for the next-ACK case, and give an RFC citation that requires it.

• PR body, "Gates at a700899ee": no guest suite has been run at this head. The guest's netstack runs ack() and duplicate() on every TCP segment in netstack_streams, netstack_streams_e1000e, netstack_socket_churn and libc_sockets. Under Evidence, every guest test the change reaches must be green at that head. The argument that slirp offers no SACK, so no probe is ever scheduled, is sound reading. But it is reading of a binary that changed, and one cargo test settles it. A change in a recovery path is no exception. Run the suite at the head that follows the change above, with command, EXIT, totals and host load.

NOTE

• PR body: three claims are false of the tree or the logs.
  • "as Linux's tcp_enter_recovery takes none in CWR", "Linux's guard" and "the brief's 'skip the TLP reduction'": see the first BLOCKER.
  • red-before.log is cited as EXIT=101, but the log has no EXIT line.
  • crate-tests.log has neither an EXIT line nor a first line naming the head. ci-host.log (EXIT=0, 78 step(s), all green, first line a700899ee) is the gate that carries this head.

SEND BACK

[Japabu]

Round 5 negative control, cut1: round 4's cut flag re-applied at 49dca4e02 (git diff 6df8222c1 a700899ee -- toyos-net-shard/tcp/src/conn.rs). git apply --check, applied, cargo test --offline --no-fail-fast in toyos-net-shard/tcp: EXIT=101, only rfc_8985_7_4_an_ack_that_infers_the_probes_loss_and_enters_recovery_cuts_twice red, left: (5068, 5068), right: (4054, 4054); git apply -R RESTORED=0, git status --porcelain empty.

diff --git a/toyos-net-shard/tcp/src/conn.rs b/toyos-net-shard/tcp/src/conn.rs
index b2cadac0b..83e025b7e 100644
--- a/toyos-net-shard/tcp/src/conn.rs
+++ b/toyos-net-shard/tcp/src/conn.rs
@@ -646,6 +646,7 @@ impl Sync {
         // RFC 8985 §7.4.2: at or past the probe's end, a probe of new data, a D-SACK of the probe or
         // a duplicate without SACK ends the episode with nothing lost; only an ACK past the end
         // without either says a resent probe repaired a loss.
+        let mut cut = false;
         if let Some((end, resent)) = self.probe.filter(|&(end, _)| ack.at_or_after(end) && acked <= flight) {
             if !resent || dsack == Some(end) || (same && seg.options.sack_blocks().len() == 0) {
                 self.probe = None;
@@ -655,19 +656,20 @@ impl Sync {
                 self.cc.cwnd = self.cc.cwnd.min(self.cc.ssthresh);
                 self.cc.end_recovery();
                 ctx.log.count(Counter::LossProbeRecovery);
+                cut = true;
             }
         }
         if acked > 0 && acked <= flight {
             self.new_ack(seg, ack, acked, flight, ctx);
             if newly {
-                self.duplicate(ack, ctx);
+                self.duplicate(ack, cut, ctx);
             }
         } else if self.sack_ok {
             if newly {
-                self.duplicate(ack, ctx);
+                self.duplicate(ack, cut, ctx);
             }
         } else if flight > 0 && seg.payload.is_empty() && !seg.syn() && !seg.fin() && ack == self.tx.una && window == self.tx.wnd {
-            self.duplicate(ack, ctx);
+            self.duplicate(ack, cut, ctx);
         }
         if ack.at_or_after(self.tx.una) && (self.tx.wl1.before(seg.seq) || (self.tx.wl1 == seg.seq && self.tx.wl2.at_or_before(ack))) {
             self.tx.wnd = window;
@@ -788,8 +790,9 @@ impl Sync {
         }
     }
 
-    /// A duplicate acknowledgment: RFC 5681 §2's without SACK, RFC 6675 §2's with it.
-    fn duplicate(&mut self, ack: Seq, ctx: &mut Ctx<'_>) {
+    /// A duplicate acknowledgment: RFC 5681 §2's without SACK, RFC 6675 §2's with it. `cut` says
+    /// this ACK already reduced cwnd for the loss a resent probe repaired.
+    fn duplicate(&mut self, ack: Seq, cut: bool, ctx: &mut Ctx<'_>) {
         let smss = self.smss();
         match &mut self.recovery {
             Recovery::Fast { inflations } => {
@@ -818,7 +821,10 @@ impl Sync {
         (self.probe_at, self.probe_due, self.probe) = (None, false, None);
         self.arm(ctx.now);
         let flight = self.tx.flight();
-        self.cc.on_loss(flight.saturating_sub(self.lt_bytes));
+        // One ACK is one congestion event, cut once, as Linux's `tcp_enter_recovery` in CWR.
+        if !cut {
+            self.cc.on_loss(flight.saturating_sub(self.lt_bytes));
+        }
         self.recover = self.tx.nxt.sub(1);
         self.episode = true;
         self.urgent = Some(self.tx.una);

[Japabu]

Review of #820, round 5, at 49dca4e02. This round's own change is e7efa0a87, followed by a merge of origin/main at 2c8a647eb.

Net lines. The whole branch against origin/main is 13 files, +824/−67. This round, against round 3's 6df8222c1, git diff 6df8222c1 49dca4e02 -- toyos-net-shard userland/netstack changes only recovery.rs: tests +20/−1, production 0. Round 4's production change of +12/−6 is gone.

Earlier BLOCKERs

• Round 4, the cut flag rests on a false oracle: CLOSED.
  • conn.rs at 49dca4e02 is the same as at 6df8222c1. duplicate(ack, ctx) is restored at conn.rs:663,667,670, and recovery entry calls on_loss unconditionally (:808).
  • rfc_8985_7_4_an_ack_that_infers_the_probes_loss_and_enters_recovery_cuts_twice asserts (4054, 4054) = 0.7 × (nxt 22721 − una 16929). Its doc cites tcp_process_tlp_ack → tcp_try_keep_open → tcp_enter_recovery.
  • Negative control mut-cut-once.log: the head line is 49dca4e02, then APPLIED. EXIT=101. Exactly one red, recovery.rs:498, left: (5068, 5068), right: (4054, 4054). RESTORED=0, porcelain empty.
• Round 4, no guest suite at the head: CLOSED.
  • guest.log opens with head 49dca4e02… and cmd: cargo test. It reads 44 passed, 44 total and EXIT=0.
  • Every guest test this branch reaches passed: netstack_streams, netstack_streams_e1000e, netstack_socket_churn, libc_sockets and https_fetch (An unchanged ureq and rustls client on a ring fork fetches byte-exact over TLS 1.3 in a guest, trusting the Mozilla roots every image carries with their licence's text #810's, which also runs over this stack).
  • Load averages were 66.92 before and 52.17 after.
• Round 1, the T14 reading at the head being landed: stands for 49dca4e02.
  • The reading was taken at 30c6678ec. git diff 6df8222c1 30c6678ec -- toyos-net-shard userland/netstack holds only the measurement-only commit's printing: rx.rs, stack.rs, lib.rs, lease/tcp.rs, streams.rs, main.rs and serve.rs.
  • git diff 6df8222c1 49dca4e02 on those paths is recovery.rs alone. So the production code that reading ran is byte-identical to this head's.
  • The merged #810, #816 and #823 touch none of the network path's code.
• Round 4's NOTEs (the body's false Linux claims, red-before.log without EXIT, crate-tests.log without head or EXIT): CLOSED.
  • The body now states the two-cut sequence and withdraws both logs.
  • The round-5 crate-tests.log has its head line and EXIT=0. Its per-binary totals add up to 406.
  • ci-host.log has its head line, 78 step(s), all green and EXIT=0. build-only.log has EXIT=0.

BLOCKER

None.

NOTE

None.

LAND