Skip to content

chore(deps): bump @effectstream pins 0.103.1 → 0.104.3 (trustProxy fix for preprod) - #76

Merged
acedward merged 1 commit into
mainfrom
00043-main-runtime-0.104.3
Sep 11, 2026
Merged

acedward merged 1 commit into
mainfrom
00043-main-runtime-0.104.3

Conversation

@acedward

@acedward acedward commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Bumps every @effectstream/* pin on the main line (Node 1 / ledger-v8) from 0.103.1 to 0.104.3, so this line picks up the runtime trustProxy fix. This is the main-line counterpart of #75 (ledger-v90.200.6); together they put the fix on both lines. Last step before the preprod kernel image can be rebuilt and the two-IP check run (SC-001).

Project 00043-rate-limit-shared-bucket, phase P6b. Follows #73 (kernel default 600/min, merged) and effectstream #925 / #926 (the runtime change and its midnight-1 port).

Why

@effectstream/runtime@0.103.1 creates Fastify with no trustProxy, so behind preprod's nginx every client collapses into a single request.ip and therefore a single rate-limit bucket. One browser tab — or the solver console's poller — exhausts the budget and locks everyone else out with 429 RATE_LIMITED. 0.104.3 (published today from effectstream midnight-1, npm dist-tag midnight-1) carries the fix.

Verified against the published tarballs, not assumed:

0.103.1 src/api/http-server.ts:171 fastify({ routerOptions: { maxParamLength: 300 } }) — no trustProxy at all
0.104.3 src/api/http-server.ts:197-199 const serverOptions: FastifyServerOptions = { …, trustProxy: parseTrustProxy(ENV.EFFECTSTREAM_TRUST_PROXY) }, with parseTrustProxy exported at :68
0.104.3 @effectstream/utils src/config.ts:210 the EFFECTSTREAM_TRUST_PROXY config entry

Scope

17 distinct packages, 31 dependency lines, 11 of the 14 package.json files. All 17 confirmed present on npm at 0.104.3 before any edit (npm view @effectstream/<name>@0.104.3 version).

wallets, celestia, midnight-contracts, orchestrator, batcher-sdk, utils, runtime, db, config, sm, concise, coroutine, event-client, log, npm-midnight-indexer, npm-midnight-node, npm-midnight-proof-server.

@effectstream/mip-zswap-offer@0.3.0 is untouched (9 lines).

Lockfile assertions

Asserted mechanically, not eyeballed:

Assertion Result
ledger-v9 entries in bun.lock 0
ledger-v8 entries 26, unchanged — this line stays on Node 1 / ledger-v8
@effectstream/*@0.103 0 (no 0.103 substring survives anywhere in the lockfile)
@effectstream/*@0.200 0
@effectstream/*@0.104.3 20 distinct names (the 17 direct + transitive crypto, event-server, sync) over 51 occurrences — exactly the 51 0.103.1 occurrences the base lockfile had

No wallet-SDK drift. The resolved @midnight-ntwrk/* (17 entries) and @midnightntwrk/* (13 entries) sets are byte-identical before and after: ledger-v8@8.1.0, onchain-runtime-v3@3.0.0, compact-runtime@0.16.0, midnight-js-*@4.1.1, wallet-sdk-facade@4.1.0, dust-wallet@4.2.0, shielded@3.0.2, unshielded-wallet@3.1.0, capabilities@3.3.1, hd@3.0.3, abstractions@2.1.0, address-format@3.1.2. No Node-2 pin appears.

One third-party transitive move, fully attributed. Diffing all 20 packages' published dependencies at 0.103.1 vs 0.104.3 (with @effectstream entries filtered and keys sorted) leaves exactly two non-@effectstream range changes: @effectstream/celestia moved @xhmikosr/bin-wrapper ^5.0.0^13.2.0, and @effectstream/sm added @noble/hashes ^2.2.0 (already resolved at 2.2.0, no lockfile change). The bin-wrapper major accounts for the rest of the lockfile diff — the Celestia binary downloader tree (@xhmikosr/downloader 9 → 15.2.0, decompress 5 → 10.2.2, got 11 → 13, file-type 12 → 20, tar-stream 1 → 3, arch 2 → 3 and their sub-trees). Upstream's own bump, confined to the Celestia binary download path.

Unlike the 0.200.6 bump on ledger-v9 (#75), ntp-time-sync stays at 0.5.0 here, so the deliberate pin guard in solver-offerfiles-real-ntp-responder.test.ts:502 needed no change.

Gates

Run in a space-free copy — the workspace path contains a space, which breaks Bun child-spawn tests.

Gate Result
bun install --frozen-lockfile on a fresh copy pass (2185 packages, 4.7 s); bun.lock unchanged
bun run typecheck (backend + solver + price-feed) 0 diagnostics in all three gates; out-of-gate dependency diagnostics 23 / 24 / 23, identical to the 24150bb baseline measured head-to-head
bun run check:pgtypes pass — generated output current, query names unique, tree stays clean
bun run check:compact-runtime n/a — the script does not exist on this line (ledger-v9 only)
CI unit set (the 10 ci.yml paths) 1465 pass / 0 fail / 14 180 expect, 1465 tests across 111 files — matching the 111 *.test.ts files those paths contain, so nothing was skipped
solver-offerfiles-real-image-secret-scan (known host-flaky) did not fail — 9 pass / 0 fail standalone as well
trade-data.test.ts "Connection terminated" (known flake) did not occur — 8 pass / 0 fail standalone; no rerun needed

Not a breaking change for this repo

No source or config file changes — dependency pins and the lockfile only. The behaviour change lives in the runtime and was flagged on effectstream #925 / #926: EFFECTSTREAM_TRUST_PROXY defaults to true (trust every hop), which is what makes request.ip the real client behind a proxy. A deployment that exposes the runtime with no proxy in front should set EFFECTSTREAM_TRUST_PROXY=false, otherwise a client can spoof X-Forwarded-For and dodge the limiter. Every known deployment fronts the runtime with nginx or Caddy.

Owner recipe — preprod redeploy and the two-IP check (SC-001)

Not done in this PR; it needs host access. After this merges:

  1. Rebuild the preprod kernel image from this merge commit and redeploy it, so the running kernel resolves @effectstream/runtime@0.104.3. Use a unique IMAGE_TAG for the build.
  2. Confirm the API nginx server block forwards the client address. The block fronting preprod.api-zswap.zkdojo.com must contain:
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    Without it the runtime has nothing to trust and the bucket stays shared — the proxy config is a deployment precondition, not part of this change. (The stack's intents-ui block already sets it; the API block was not in the workspace and has to be checked on the host.)
  3. Leave EFFECTSTREAM_TRUST_PROXY unset on preprod — the true default is what is wanted behind nginx.
  4. Exhaust the budget from one IP:
    for i in $(seq 1 700); do
      curl -s -o /dev/null -w '%{http_code} ' \
        https://preprod.api-zswap.zkdojo.com/v1/market/pairs
    done; echo
    Expect 200s until the limit, then 429. Confirm the body is {"statusCode":429,"error":"RATE_LIMITED","reason":…} and check the headers:
    curl -sD - -o /dev/null https://preprod.api-zswap.zkdojo.com/v1/market/pairs | grep -i x-ratelimit
    x-ratelimit-limit should read 600 (the feat(api): raise the default per-IP rate limit to 600/min #73 default; deploy/ sets an explicit 6000 if that env is in play).
  5. From a second IP (phone on cellular, or any host on a different egress address), request the same route immediately:
    curl -sD - -o /dev/null https://preprod.api-zswap.zkdojo.com/v1/market/pairs | grep -i x-ratelimit
    Pass: 200 with its own x-ratelimit-remaining, close to the full budget rather than 0 — proving the two IPs now have independent counters. Fail: 429, or a remaining that continues IP A's count — which means step 2's header is still missing.

Use /v1/market/pairs or any counted route; /v1/health* and the docs routes are exempt from the limiter and will not move the counter.

CI

All four jobs green on the runner (ubuntu-latest, pinned Bun 1.3.3) — run 34596197114: Unit tests (bun test, PGlite) 2m16s, Backend typecheck 31s, Solver typecheck 31s, Typecheck docs playground 26s. e2e skipped (workflow_dispatch-only).

One open question for the reviewer

The one thing this bump changes that no gate on this line covers: @effectstream/celestia@0.104.3 moved @xhmikosr/bin-wrapper ^5.0.0^13.2.0, and that is the component which lazily downloads celestia-appd / celestia into the package's vendor/ dir. The CI unit set never downloads a binary, and the only job that would (e2e) is manual and needs live chain infra — so this PR is green while the swapped code path stays unexercised. scripts/patch-macos-celestia.ts (the predev/pretest step) is also coupled to that path, since it patches the Mach-O binaries bin-wrapper leaves in vendor/.

Mitigating evidence: @effectstream/celestia's index.js is byte-identical at 0.103.1 and 0.104.3 — the same new BinWrapper().src(…).dest(dest).use(…) calls, the same pinned binaries (celestia-appd 6.4.10, celestia-node 0.28.4) from the same source (effectstream/binaries release 0.3.120). Only the downloader implementation underneath changed.

Suggested cheap check before merging, on Linux: delete node_modules/@effectstream/celestia/vendor and force a .use() (e.g. bunx @effectstream/celestia start-node --help), confirming both binaries land and report their versions. If that is inconvenient, the preprod rebuild in step 1 above should be watched for a Celestia fetch failure rather than assumed routine. Recorded as Q6 in the project's questions file.

…Proxy fix

The kernel's `main` line (Node 1 / ledger-v8) pinned every @effectstream
package at 0.103.1. That runtime builds Fastify with no `trustProxy`, so
behind preprod's nginx every client collapses into one `request.ip` and a
single shared rate-limit bucket: one browser tab (or the solver console's
poller) locks everyone else out with 429 RATE_LIMITED.

0.104.3 was published today from effectstream `midnight-1` (dist-tag
`midnight-1`) and carries the runtime fix from effectstream #925/#926:
`EFFECTSTREAM_TRUST_PROXY` parsed into Fastify's `trustProxy`, so
`request.ip` becomes the real client again and the limiter counts per
client. Verified against the published tarballs rather than assumed:

  0.103.1  src/api/http-server.ts:171
           fastify({ routerOptions: { maxParamLength: 300 } })    <- no trustProxy
  0.104.3  src/api/http-server.ts:197-199
           const serverOptions: FastifyServerOptions = {
             ...,
             trustProxy: parseTrustProxy(ENV.EFFECTSTREAM_TRUST_PROXY),
           }
  0.104.3  @effectstream/utils src/config.ts:210  EFFECTSTREAM_TRUST_PROXY

This is the `main`-line counterpart of the `ledger-v9` bump to 0.200.6
(PR #75); together they put the fix on both lines. It is the last step
before the preprod kernel image can be rebuilt and the two-IP check run.

Scope: 17 distinct @effectstream packages, 31 dependency lines across 11
of the 14 package.json files -- wallets, celestia, midnight-contracts,
orchestrator, batcher-sdk, utils, runtime, db, config, sm, concise,
coroutine, event-client, log, npm-midnight-indexer, npm-midnight-node,
npm-midnight-proof-server. All 17 confirmed present on npm at 0.104.3
before editing. @effectstream/mip-zswap-offer@0.3.0 is untouched.

Lockfile, asserted rather than eyeballed:
  - ledger-v9 entries: 0; ledger-v8 entries: 26, unchanged. This line
    stays on Node 1 / ledger-v8 and does not drift toward Node 2.
  - @effectstream/*@0.103: 0 (no `0.103` substring survives anywhere);
    @effectstream/*@0.200: 0.
  - 20 @effectstream names now at 0.104.3 (the 17 direct plus transitive
    crypto, event-server, sync) over 51 occurrences -- exactly the 51
    occurrences 0.103.1 had.
  - The resolved @midnight-ntwrk/* (17 entries) and @midnightntwrk/*
    (13 entries) sets are byte-identical before and after: ledger-v8
    8.1.0, onchain-runtime-v3 3.0.0, compact-runtime 0.16.0, midnight-js
    4.1.1, wallet-sdk facade 4.1.0 / dust 4.2.0 / shielded 3.0.2 /
    unshielded 3.1.0 / capabilities 3.3.1 / hd 3.0.3 / abstractions
    2.1.0 / address-format 3.1.2. No wallet-sdk drift.

One third-party transitive move, fully attributed: diffing all 20
packages' published `dependencies` at 0.103.1 vs 0.104.3 leaves exactly
two non-@effectstream range changes -- @effectstream/celestia moved
@xhmikosr/bin-wrapper ^5.0.0 -> ^13.2.0, and @effectstream/sm added
@noble/hashes ^2.2.0 (already resolved at 2.2.0, no lockfile change).
The bin-wrapper major accounts for the rest of the lockfile diff: the
Celestia binary downloader tree (@xhmikosr/downloader 9 -> 15.2.0,
decompress 5 -> 10.2.2, got 11 -> 13, file-type 12 -> 20, tar-stream
1 -> 3, arch 2 -> 3 and their sub-trees). Upstream's own bump, confined
to the Celestia binary download path.

Unlike the 0.200.6 bump on ledger-v9, ntp-time-sync stays at 0.5.0 here,
so the deliberate pin guard in solver-offerfiles-real-ntp-responder.test.ts
needed no change.

Gates, in a space-free copy (the workspace path contains a space):
  bun install --frozen-lockfile on a fresh copy  pass, lockfile unchanged
  bun run typecheck                              0 diagnostics in all
                                                 three gates; out-of-gate
                                                 23/24/23, identical to
                                                 the 24150bb baseline
  bun run check:pgtypes                          pass, tree stays clean
  CI unit set (10 ci.yml paths)                  1465 pass / 0 fail /
                                                 14180 expect, 111 files
check:compact-runtime does not exist on this line (ledger-v9 only).
@acedward
acedward marked this pull request as ready for review September 11, 2026 12:12
@acedward
acedward merged commit 8aa8abe into main Sep 11, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant