Skip to content

feat: throttle outgoing bandwidth and redirect saturated serves to peers#3

Merged
MichaelTaylor3d merged 1 commit into
mainfrom
feat/outgoing-bw-throttle-30
Jul 4, 2026
Merged

feat: throttle outgoing bandwidth and redirect saturated serves to peers#3
MichaelTaylor3d merged 1 commit into
mainfrom
feat/outgoing-bw-throttle-30

Conversation

@MichaelTaylor3d

Copy link
Copy Markdown
Contributor

Summary

  • Adds a configurable outgoing-bandwidth cap (DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC, unlimited by default) on the node's serve path (new bandwidth module, OutgoingThrottle: a fixed-1s-window byte budget).
  • When serving would push the node's outgoing traffic over the cap, it resolves alternate holders via the DHT (download::NodeContent::find_providers) and answers with the same CONTENT_REDIRECT (-32008) shape + hop-cap discipline the existing #165 redirect-on-miss mechanism already uses — extending redirect from "not held" to "held but saturated," per dig_ecosystem issue #30.
  • Graceful fallback: when no alternate holder is known (or no P2P engine is attached — the in-process FFI/DIG-Browser path), the node serves the request anyway. The throttle changes where a request is served from when it can, never whether — nothing is dropped or fails closed.
  • Wired into the three surfaces that return locally-held resource bytes: dig.getContent's local-first serve (cold cache hit + post-§21-sync hit), dig.fetchRange, and the mTLS peer range-stream (stream_range — the busiest node-to-node egress path, since multi-source downloaders fan byte-ranges across it).
  • providers[].addresses inherit the DHT's IPv6-first candidate ordering (§5.2) — the throttle does not reorder them.

Blast radius (gitnexus impact analysis, §2.0)

Ran npx gitnexus@latest analyze on the repo, then gitnexus impact (CLI) on the touched symbols before editing:

  • Node struct: exactly 5 direct construction sites, all in crates/dig-node-core/src/lib.rs (the from_env production constructor + 4 test constructors) — all updated with the new outgoing_throttle field.
  • fetch_range_frame: 2 real call sites (lib.rs dispatch, peer.rs::NodeResponder::stream_range) — both updated.
  • stream_range: a trait method (PeerRpcResponder) with independent per-type impls (StubResponder, integration-test responders) — editing the real NodeResponder impl does not affect the others.
  • redirect_error_object: 1 existing caller — my new call sites are additive.
  • Cross-crate check (grep across dig-node-service/dig-runtime): both only consume Node opaquely + the free functions handle_rpc/cache_cap_bytes/cache_used_bytes — none of the signatures touched here cross the crate boundary. Risk contained to dig-node-core (lib.rs, peer.rs, new bandwidth.rs).
  • gitnexus detect-changes --scope compare --base-ref main post-commit reports "critical" risk / 71 affected flows — this is the Node struct's inherent connectivity (touching its private-field literal necessarily touches the 5 constructor sites and everything the two gap_fill/chain_watch end-to-end tests exercise), not a functional regression: the full workspace test suite (below) is green.

How verified

TDD throughout (failing test first, then implementation):

  • bandwidth.rs: 8 new unit tests for the throttle's window/env-parsing logic (pure, no sleeps — *_at variants take the instant explicitly).
  • lib.rs: 6 new integration tests over the real dig.getContent/dig.fetchRange dispatch — cap-exceeded+provider→redirect, no-provider fallback, no-P2P-engine fallback, under-cap non-interference, and hop-cap reuse (verified genuinely RED before wiring the dispatch, then GREEN after).
  • peer.rs: 2 new tests over the real NodeResponder::stream_range (via tokio::io::duplex + serve_one_stream) — over-cap+provider→redirect frame, no-provider→streams anyway.
  • cargo fmt --all -- --check: clean.
  • cargo clippy --workspace --all-targets -- -D warnings: clean.
  • cargo build --workspace: clean.
  • cargo test -p dig-node-core (lib + tests/dht_integration.rs + tests/peer_network.rs + doctests): 209 + 8 + 2 passed, 0 failed.
  • cargo test --workspace (dig-node-service, dig-runtime, dig-wallet): all green, no regressions.

Docs / SPEC

  • SPEC.md: new §17 "Outgoing-bandwidth throttle and redirect-on-saturation" (config, accounting, serve-path integration, hop-cap reuse, graceful fallback); DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC added to the §3.2 variables table; -32008 CONTENT_REDIRECT added to the §10 error catalogue (previously undocumented despite already being emitted by #165 redirect-on-miss).
  • README.md: new env-var row.
  • docs.dig.net peer-network page is not updated in this PR per the task's "ONLY dig-node; never another repo" scope constraint (and that repo currently has unrelated in-flight uncommitted changes on another branch) — flagging as a follow-up for whoever next touches docs.dig.net's peer-network/error-catalogue pages.

Version bump

crates/dig-node-core/Cargo.toml: 0.2.00.3.0 (minor — a new, backwards-compatible, opt-in capability: feat:, matching this repo's precedent, e.g. 0.1.00.2.0 for the dig.getManifest feature). The workspace root Cargo.toml has no [package]/[workspace.package] version field, so the ensure-version-increment gate is a no-op there per its own documented behavior.

Test plan (for reviewers)

  • cargo test -p dig-node-core green (209 lib + 8 + 2 integration)
  • cargo test --workspace green
  • cargo fmt --all -- --check clean
  • cargo clippy --workspace --all-targets -- -D warnings clean
  • cargo build --workspace clean
  • CI required checks (fmt / clippy / test+coverage / commitlint / version-increment) — confirming green after opening

Closes/references DIG-Network/dig_ecosystem#30.

Add a configurable outgoing-bandwidth cap (DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC,
unlimited by default) on the node's serve path. When serving would push this
node's outgoing traffic over the cap, it resolves alternate holders via the DHT
and answers with the SAME CONTENT_REDIRECT (-32008) shape and hop-cap discipline
the existing #165 redirect-on-miss mechanism uses, instead of serving over-budget.
When no alternate holder is known (or no P2P engine is attached), it serves the
request anyway - a throttle that redirects when it can, never one that drops or
fails closed.

Wired into the three surfaces that return locally-held resource bytes: the
dig.getContent local-first serve, dig.fetchRange, and the mTLS peer range-stream
(the busiest node-to-node egress path multi-source downloaders fan across).

New bandwidth module with a fixed-1s-window throttle (unit-tested), plus
integration tests proving cap-exceeded-with-provider redirects, no-provider and
no-P2P-engine graceful fallback, under-cap non-interference, and hop-cap reuse.
SPEC.md gains a new outgoing-bandwidth-throttle section, the DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC
variable, and the -32008 error-catalogue entry (previously undocumented).
@MichaelTaylor3d MichaelTaylor3d merged commit c3c5956 into main Jul 4, 2026
5 checks passed
@MichaelTaylor3d MichaelTaylor3d deleted the feat/outgoing-bw-throttle-30 branch July 4, 2026 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant