Skip to content

relay: TCP listener, framing encoders and a stdlib Python client for the sidecar stream - #12

Merged
CSSFrancis merged 13 commits into
directelectron:mainfrom
TheDrOnos:feat/relay
Sep 25, 2026
Merged

CSSFrancis merged 13 commits into
directelectron:mainfrom
TheDrOnos:feat/relay

Conversation

@TheDrOnos

Copy link
Copy Markdown
Contributor

relay: TCP listener, framing encoders and a stdlib Python client for the sidecar stream

Summary

Adds the transport pieces an app needs to expose its backend stream over TCP, with no
policy of their own:

  • de_shell/js/main/framing.ts — encodeMessage / encodeBinary, the inverse of
    stdoutDemux.ts. Byte-identical framing to ipc.emit and anyplotlib's encode_frame.
  • de_shell/js/main/relay.ts — createRelay({host, port, onConnection, onLine, onClose, …}).
    Binds exactly the given address (an empty host is refused; 0.0.0.0 only when asked),
    hands each client line to the app as a string (the first one included, so the app owns
    admission), guards every connection until the app calls admit() (hello timeout, a
    small pre-admission line cap, a larger one after), writes framed units per connection
    with the payload as its own chunk, exposes writableLength for the app's backpressure
    decisions, closes gracefully so a written refusal reaches the client, and sets keepalive
    and no-delay on accepted sockets. Imports only node:net, so it runs under node --test.
    A createServer factory option leaves room for TLS without an API change.
  • de_shell/remote_client.py — a synchronous, stdlib-only client: connect, send,
    recv (("message", dict) / ("binary", header, bytes) / ("stream", str)), close.
    recv waits on a selector so a send on another thread is never cut short. Malformed
    units raise ValueError with the offending bytes on .line, after being consumed.
  • Barrel re-exports; tests/test_boundary.py gains the new module; one towncrier fragment.

Nothing in backendProcess.ts or stdoutDemux.ts changes. Existing apps are unaffected
until they call the new exports.

Consumers

  • Ground Crew's remote-control endpoint (directelectron/de_ground_crew#321): pairing,
    tokens, admission, the action filter and snapshots all stay in GC; the endpoint is a thin
    use of createRelay, and GC's laptop-side connector uses remote_client.
  • spyde #159 (backend on a cluster, GUI on the laptop): the same encoders and client are
    the codec for its serve process and connect mode; both stay yours.

Deliberately not in this PR

Connect mode for Electron, a Python serve relay, a subscriber API on backendProcess
(Ground Crew tees in its own handlers), any auth or policy, TLS itself (only the factory
hook), an onServerError hook for post-listen server errors (currently swallowed so
EMFILE cannot crash main — worth adding when an app wants the log line).

Two calls left to you: whether host should additionally require an IP literal (a
hostname is DNS-resolved to whichever address comes first); and, with TLS, relay.close()
would wait for sockets still in handshake, which the relay never sees — a doc note or a
close timeout when the hook is first used.

Tests

  • framing.test.ts (8): encoders decoded through the production demuxer, at several chunk
    sizes, with marker look-alikes and \n inside payloads, non-ASCII headers, non-finite
    numbers refused.
  • relay.test.ts (32): real loopback sockets on port 0 — exact bind and OS-error rejection,
    empty host refused, lines across chunk splits and \r\n, hello timeout and both caps on
    an injected clock, two clients delivered (no count of its own), writableLength under a
    stalled reader, graceful close delivering a queued refusal before EOF, destroy after the
    grace, relay.close() races, keepalive / no-delay / createServer injection observed
    through a spy factory.
  • tests/test_remote_client.py (20): the decoder fed real ipc.emit / encode_frame bytes
    over a socketpair, whole and at 1- and 7-byte steps; recv timeout keeps partial bytes;
    EOF → ConnectionError; malformed prefix/header/JSON → ValueError with .line; a poll
    on one thread beside a blocking send on another; and a cross-runtime round trip against a
    real createRelay under Node (skips by name when node is absent or cannot strip types).

Evidence

  • Windows (Node 24.16.0, Python 3.14): npm run typecheck clean · npm run test:unit
    133/133 · pytest 117 passed, 0 skipped · wheel carries the three new source files.
  • Linux (Node 24.19.0, Python 3.13): npm run typecheck clean · npm run test:unit
    133/133 · pytest 117 passed, 0 skipped. (npm 11 withheld electron's postinstall under
    allow-scripts; neither suite needs the binary.)

Release

New public API on both halves; a 0.4.0 release would let apps pin it
(de-shell>=0.4,<0.5). The fragment is upcoming_changes/+relay.new_feature.rst; it will
be renamed to <this PR's number>.new_feature.rst once the number exists.

@CSSFrancis
CSSFrancis merged commit 5329614 into directelectron:main Sep 25, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants