Skip to content

test(interop): go-libp2p WebRTC-Direct interop tests - #1471

Merged
acul71 merged 29 commits into
libp2p:mainfrom
yashksaini-coder:feat/webrtc-go-interop-tests
Sep 6, 2026
Merged

acul71 merged 29 commits into
libp2p:mainfrom
yashksaini-coder:feat/webrtc-go-interop-tests

Conversation

@yashksaini-coder

Copy link
Copy Markdown
Contributor

The formal go-libp2p interop tests for #1437.

A small pinned go-libp2p v0.49 harness (tests/interop/go_libp2p/webrtc_direct/, listen / dial -version N) is built on demand under a file lock and skipped when the Go toolchain is missing, then driven against our WebRTC-Direct transport in both directions and both versions.

Best on top of #1459 (needs the v2 dialer + framing fixes), but the harness/tests stand alone. go → py is a real listener-side bug, not something this PR introduces.

Refs #1437

🤖 Generated with Claude Code

yashksaini-coder and others added 17 commits August 28, 2026 11:52
…rity by DTLS role

go-libp2p (go-msgio pbio fallback) writes the uvarint prefix and the
protobuf body as two SCTP messages, so every py<->go WebRTC-Direct
connection died at Noise msg#1 with "malformed handshake frame length".
New stream._decode_frames pops complete frames off an accumulating
buffer (frames may be split or batched across SCTP messages, bounded by
MAX_MESSAGE_SIZE); DataChannelReadWriter and WebRTCStream.on_data both
use it. A malformed frame now resets the stream instead of dropping one
message and desynchronising.

With the handshake passing, a py listener then hit aiortc's
`assert stream_id not in self._data_channels`: aiortc picks DCEP stream
id parity from the ICE role, which is inverted for a WebRTC-Direct
listener (ICE-controlled, DTLS server), so it used even ids like go's
dialer. _create_channel now passes an explicit id by DTLS role
(RFC 8832: client even, server odd).

Verified live against go-libp2p v0.49.0: go->py and py->go, v1 and v2.

Refs libp2p#1437
- keep aiortc allocating (and recycling) SCTP stream ids; only re-seed the
  parity from the DTLS role (explicit ids tied the 16-bit space to our
  never-recycled counter)
- on_data: deliver frames decoded before a malformed one, reset the peer
  (RESET + cleanup) instead of a local-only reset, ignore bytes after reset,
  and apply a whole SCTP message in one trio hop
- _decode_frames: derive the max prefix length from MAX_MESSAGE_SIZE

Refs libp2p#1437
The test performs two full dials and two bounded peer-connection closes;
on Windows each close can take ~5 s when a datagram write is in flight, so
30 s was flaky (seen once on CI, no socket errors logged).

Refs libp2p#1437
… collides

Harness mode binds TCP on the same port number as the OS-chosen UDP mux
port; Windows reserves port ranges per protocol, so that TCP bind can fail
with WinError 10013 (seen on CI in the harness loopback test). Retry up to
five times with a new UDP port before giving up; explicit ports still fail
loudly.

Refs libp2p#1437
…epts v2, dialer opt-in

Listener: stop dropping libp2p+webrtc+v2/ first contacts. The server ufrag
minus the prefix is the dialer's ICE pwd (must be a valid one, 22..256
ice-chars, else rejected in parse_direct_username); the inferred offer uses
client_ufrag + that pwd, and our local ufrag/pwd stay server_ufrag verbatim
as in v1. _accept_v1 -> _accept(client_pwd=...).

Dialer: new WebRTCTransportConfig.webrtc_direct_dial_version (default 1).
v2 keeps the aioice ufrag/pwd (no munging) and sets the synthetic answer's
ufrag == pwd to "libp2p+webrtc+v2/" + local_password. Default stays v1 while
specs#715 is unmerged; unknown values raise WebRTCConnectionError.

Tests: v2 parse/reject cases, SDP shape, v2 loopback echo, no-munge check,
short-suffix rejection, mixed v1+v2 dialers on one listener.

Refs libp2p#1437
- validate webrtc_direct_dial_version in the config (was only checked
  mid-dial after the PC was built)
- parse_direct_username returns the validated client pwd; make_v2_credential;
  one pwd check instead of two version branches
- inline the v1 credential writes; trim the dial() comment

Refs libp2p#1437
… RESET guards

- _create_channel: a missing RTCDtlsTransport._role now degrades to
  aiortc's default stream-id parity with a warning instead of asserting
  (an assert in a live connection path, also stripped under -O)
- _decode_frames: reject consumed == 0 from the varint decoder rather
  than silently indexing head[-1]
- _reset_locally is idempotent; _apply_batch_on_trio_thread stops
  delivering a batch's remaining frames once the stream is RESET
- note in DataChannelReadWriter._fill that frames after FIN/RESET are
  dropped by design; wrap the newsfragment

Refs libp2p#1437
Review follow-up on libp2p#1459: README readers should see that libp2p/specs#715
is unmerged and cross-implementation v2 interop is not yet guaranteed
(config.py already said so).

Refs libp2p#1437
A small pinned go-libp2p v0.49 harness (listen / dial -version N), built on
demand under a lock and skipped when the Go toolchain is missing, drives our
WebRTC-Direct transport against a real go peer. py -> go is verified live for
v1 and v2; go -> py is xfail (strict) pending libp2p#1470, where the listener tears
the peer connection down right after ICE completes.

Refs libp2p#1437, libp2p#1470
@yashksaini-coder
yashksaini-coder force-pushed the feat/webrtc-go-interop-tests branch from cb9db7f to c01c64c Compare September 3, 2026 18:12
@acul71

acul71 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

🤖 Generated with Claude Code

Hi @yashksaini-coder , you used claude code with which LLM ?

@acul71 acul71 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review readiness

Thanks for the clean interop harness — scope matches the description (pinned go-libp2p v0.49 listen/dial, py→go live, go→py strict xfail for #1470), and the #1459 dependency is already on main.

Not ready to dig into a full review pass yet; a couple of blockers first:

Blockers

  1. Lint is red. Recent CI fails typecheck on test_webrtc_direct_interop.py:

    Object of class `NoneType` has no attribute `receive_some`
    

    (proc.stdout is optional after open_process). Please assert/narrow stdout (or otherwise satisfy the checker) and get lint green.

  2. Maintainer question still open — which Claude model did you use with Claude Code? (comment above)

Follow-ups (can land with or right after the lint fix)

  • Go is not installed in CI. .github/workflows/tox.yml sets up Nim/Node for interop, but not Go. Without actions/setup-go (matching go.mod / go-libp2p v0.49), these tests will skip on Actions and CI won’t actually exercise the harness.
  • Newsfragment is 1470.internal.rst (issue id); consider 1471 if that’s the project convention for the PR that adds the tests.
  • Commit history is noisy (already-merged framing/v2 work + merges); net diff vs main is fine — squash-on-merge is enough.

Happy to do a deeper review once lint is green (and ideally once Go is wired into the interop job so CI runs the new tests for real).

@yashksaini-coder

Copy link
Copy Markdown
Contributor Author

Thanks! Addressed the blockers and follow-ups:

  • Lint green — narrowed proc.stdout (it's Optional after open_process); pyrefly is now clean on the interop module.
  • Go in CI — added an actions/setup-go@v5 step (go 1.25, matching go.mod) to the interop tox job, right after the Node.js setup, so the harness actually builds and the tests run on Actions instead of skipping.
  • Newsfragment renamed 1470.internal.rst1471.internal.rst (PR id).
  • History — rebased onto current main (v2/framing already landed), so the branch is now a single clean interop commit; net diff is just tests/interop/go_libp2p/ + the CI/newsfragment bits.

Local: 2 passed (py→go v1+v2) / 2 xfailed (go→py v1+v2 → #1470). Ready for a deeper pass whenever you are.

…wsfragment

- narrow proc.stdout (Optional after open_process) so pyrefly is green
- add actions/setup-go to the interop tox job so the harness builds and
  the tests run on Actions instead of skipping
- newsfragments/1470.internal.rst -> 1471.internal.rst (PR id)

Refs libp2p#1437, libp2p#1470
@yashksaini-coder

yashksaini-coder commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Generated with Claude Code

Hi @yashksaini-coder , you used claude code with which LLM ?

I recently got hands on claude code, mainly for implementing and testing the main feature changes affect on other module features. It also helps me explain for other cases and scenraios, which helped me not to raise any more follow up PRs, after a big feature or PR lands, as I used to do before

@acul71 acul71 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review

Previous blockers are addressed:

  • Lint: proc.stdout narrowed with assert stdout is not None — lint green
  • Go in CI: actions/setup-go@v5 (go-version: '1.25') on the interop tox job; harness builds and tests run (py→go PASS v1/v2, go→py strict XFAIL → #1470)
  • Newsfragment renamed to 1471.internal.rst
  • CI currently all green; merges cleanly with current main (1 commit behind, no conflicts)

Harness design looks good: pinned go-libp2p v0.49, session fixture + file lock for xdist, skip when Go missing, both directions/versions covered. Strict xfail on go→py is the right call until #1470 lands.

Non-blocking follow-ups

  1. _read_lines drops leftover LISTEN lines. In test_py_dials_go, the first _read_lines(..., count=1) can consume a multi-line burst and discard extra LISTEN lines still in the local buffer; the second call (count=3) then sits on the 45s move_on_after timeout. That matches the ~45s py→go call times in CI and can defeat _lan_addr’s non-loopback preference. Prefer one read that collects all LISTEN lines (short settle after the first), or preserve leftover buffer across calls.
  2. Branch is 1 commit behind main — optional merge/rebase before squash; not required.
  3. Claude model question still unanswered by name — optional.

LGTM — OK to squash-merge.

acul71 and others added 2 commits September 6, 2026 21:19
Avoid a second 45s wait that dropped already-buffered LISTEN lines and
could pick loopback over LAN; one short settle drain keeps the burst.

Co-authored-by: Cursor <cursoragent@cursor.com>
@acul71

acul71 commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to the review note on _read_lines / LISTEN collection:

Pushed a fix that waits for the first LISTEN line, then briefly settles (~250ms) to drain sibling addresses from the same burst in one buffer — instead of a second count=3 wait that discarded already-read lines and burned ~45s per py→go case.

Locally: test_py_dials_go v1/v2 both PASS in ~0.3–0.4s each (was ~45s call time). Also merged latest main.

@acul71
acul71 merged commit b4343c3 into libp2p:main Sep 6, 2026
38 checks passed
@yashksaini-coder
yashksaini-coder deleted the feat/webrtc-go-interop-tests branch September 7, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants