ledger-v9

[acedward]

DO NOT MERGE UNTIL LEDGER V9 IS THE STANDARD

This is the re-proposal of #62, which was merged by mistake and reverted in #64. The branch tip reapplies #62 on top of that revert (tree identical to 7a357f8, the tip #62 merged), so the diff below is the whole ledger-v9 line against main. Kept as a DRAFT so it cannot be merged by accident.

---

Single PR replacing #49 (00001-ledger-v9 → main) and #50 (00001-solver-v9 → feat/cow-solver): this branch is 00001-solver-v9 (which already contains 00001-ledger-v9 and feat/cow-solver @ 381022d) with main @ aa35722 merged in and everything main gained since the fork ported from ledger-v8 to ledger-v9.

BREAKING, exactly as #49: the whole workspace moves to the node-2.x / ledger-v9 line (@effectstream/*@0.200.2, @effectstream/mip-zswap-offer@0.4.0-v9.0, @midnightntwrk/ledger-v9@1.0.0-rc.3, midnight-js 5.0.0-beta.6, wallet-sdk v2 line, compactc 0.33.0-rc.2). v8-era blobs and fixtures no longer deserialize. See LEDGER-V9-MIGRATION.md (this branch adds a closing section, "Unified branch: ledger-v9").

What the merge had to port

main moved a lot after 381022d — the token price service (#54–#56), the offer poster and its Compose service (#57, #60), the solver status listener and console (#58, #59), sNight seeding (#61) — all written against ledger-v8. Merged as-is that code would not even resolve, so on top of the five textual conflicts:

• Root manifest: the wallet-sdk set the offer poster added (facade 4.1.0, dust-wallet 4.2.0, shielded 3.0.2, unshielded-wallet 3.1.0, hd 3.0.3, capabilities 3.3.1, abstractions 2.1.0) moves to the versions @effectstream/midnight-contracts@0.200.2 itself resolves (5.0.0-beta.2, 5.0.0-beta.2, 4.0.0-beta.2, 4.0.0-beta.2, 3.1.0-beta.1, 4.0.0-beta.2, 3.0.0-beta.0). packages/price-feed's @effectstream/{db,utils}@0.103.1 — the last dependency still pulling ledger-v8@8.1.0 into the store — moves to 0.200.2. Lockfile regenerated: zero ledger-v8, exactly one ledger-v9@1.0.0-rc.3.
• deploy/scripts/lib/pinned-wallet.ts (the offer poster's copy of buildWalletFacade with a pinned coin selector) re-diffed against midnight-contracts@0.200.2: tagged createKeystore({kind: "schnorr", secret}), facade factories that receive their configuration, DustAddress.encodePublicKey instead of getInitialDustState + MidnightBech32m, TransactionHistoryEntryCommonSchema, setNetworkId, HD key material cleared after derivation. The selector itself is untouched.
• waitForDustFunds now resolves a readiness record; the poster reads .balance.
• @midnight-ntwrk/ledger-v8 → @midnightntwrk/ledger-v9 in the seven files main added (faucet mint, fee sizing, asset-price / batcher tests, the deploy e2e driver); two sync signData callbacks in deploy/scripts → signDataAsync.
• deploy/ Compose stack onto the 2.x chain, keeping its "same binaries as bun run dev" rule: midnight-node 2.0.0-rc.4 and indexer-standalone v4.4.0-rc.3 from binaries 0.3.120 (sha256-pinned, linux amd64 + arm64), official multi-arch midnightntwrk/proof-server:9.0.0-rc.5 pinned by digest (0.3.120 has no linux-arm64 rc.5 binary; npm-midnight-proof-server@0.200.2 falls back to the same image), the kernel image compiling with the same LFDT compactc 0.33.0-rc.2 asset infra/compact-toolchain.Dockerfile uses (via $COMPACTC → infra/compact.sh), indexer config.yaml from npm-midnight-indexer@0.200.2, /api/v4 as the default indexer path for kernel and relay.
• start.dev.ts compact-check no longer demands a compact-manager install of 0.30.0 (which the v9 build never used): it reads infra/compact-version.txt and verifies the route infra/compact.sh takes ($COMPACTC, else Docker).
• Docs: deploy/README.md provenance tables, README.md / API.md / docs/protocol-scope.md supported-domain statements → Midnight 2.x / ledger-v9.

Verification

• bun run typecheck (backend, solver, price-feed): 0 diagnostics.
• bun run check:pgtypes + the CI unit set (packages/{database,validator,node,offer-guard,batcher,solver-core,solver,solver-frontend,tests/grand-e2e}, deploy/scripts/lib): 1,893 pass / 1 fail across the sets. The one failure, real E1 physical image secret scanner > handles a large physical inventory … (a Docker-run scanner over a 16k-hardlink fixture exiting 70), reproduces identically from a clean origin/main worktree on the same host, so it is host-dependent and not introduced here.
• Docker: deploy/images/{midnight-node,indexer,proof-server} build and answer --version on arm64; the kernel image builds through the compactc 0.33.0-rc.2 step and emits src/managed/{compiler,contract,keys,zkir}: all built on linux/arm64. midnight-node --version → 2.0.0; indexer-standalone --version → 4.4.0-rc.3 (56561b2f); proof-server resolves to index digest d96a4d0f… with PORT=6300; the kernel image reports compactc 0.33.0 and carries src/managed/{compiler,contract,keys,zkir} with prover/verifier keys. (Building images/midnight-node from the repo-root context needs a trimmed context on a checkout with node_modules present — that Dockerfile has no .dockerignore, same as before this PR.)
• Not run here: the live e2e (bun run test) and a full Compose bring-up — both need a chain stack. feat!: migrate the kernel to ledger-v9 / node 2.x (BREAKING) #49 was validated live at bun run test 62/0 on its own tree; the Compose stack's first live run on 2.x is still owed, in particular the Midnight Intents reference relay service (deploy/relay, reference checkout 061f4d3) has never been run against a 2.x chain from this stack.

Inherits #48's / #50's NO-GO-for-real-funds posture unchanged. Once this lands, #49 and #50 can be closed.

[acedward]

Merged main @ 8fd63b4 (#63 — every token has 6 decimals; faucets mint whole coins) into this branch as 14782a2. 40 files, no textual conflicts; the reapply commit 544670d is kept underneath so the PR diff still shows the whole ledger-v9 line.

Audit: nothing from #63 reintroduces ledger-v8 — the lockfile has zero ledger-v8 entries, packages/solver-core/amount.ts is dependency-free, and the 14 files that overlap with the v9 port (faucet-mint.ts, poster-config.ts, mint-test-tokens.ts, api.ts, docs) merged cleanly by offset.

Gates (space-free copy, v9 lockfile installed, bun install --frozen-lockfile no changes):

Gate	Pre-merge 544670d	Merged 14782a2
bun run typecheck (backend / solver / price-feed)	—	0 diagnostics
bun run check:pgtypes	—	generated output is current
CI unit set (packages/{database,validator,node,offer-guard,batcher,solver-core,solver,solver-frontend,tests/grand-e2e}, deploy/scripts/lib)	1525 pass / 2 fail / 1 error	1551 pass / 1 fail / 1 error

The residual fail + error are the same on both sides and environmental: the copy has no compiled Compact artifacts (contract-offer-files.*.json for offer-guard.test.ts, managed/contract/index.js for p3b-closeout.test.ts) — compactc 0.33.0-rc.2 was not run here. Not a live e2e or Compose run, same as before.

[acedward]

00027 (randomised give size) merged into ledger-v9 — 4af1025

00027-offer-poster-size-range (kernel PR #66, open against main) is now on this branch, so #65 will not regress the poster's give-size range when it lands. Same precedent as the #63 merge (14782a2).

git merge --no-ff origin/00027-offer-poster-size-range → 4af1025, 13 files, +933/−9, no textual conflicts (confirmed with a git merge-tree dry run first). main had not moved since 14782a2 merged it (still 8fd63b4), so no second merge was needed.

The v9 line's own poster deltas are intact — both sit outside the regions 00027 touches, and were re-checked after the merge, not assumed:

v9 delta	State after the merge
poster-config.ts indexer endpoints on api/v4	4 occurrences, unchanged
offer-poster.ts waitForDustFunds returning a readiness record (dustFunds.balance, 0.200.x)	unchanged

No manifest or lockfile is touched (package.json/bun.lock not in the diff) and the merge introduces no ledger-v8 / 0.103.x / 0.104.x pin — 00027 adds no dependency; poster-size.ts is self-contained.

Gates on this branch (space-free copy of the clone at 4af1025, bun install --frozen-lockfile → 2188 packages, bun 1.3.11):

Gate	Result
bun test deploy/scripts/lib	271 pass / 0 fail, 3572 expect() calls, 10 files
bun run typecheck	0 diagnostics — backend, solver and price-feed
bun test packages/solver/launch.test.ts (the one suite that reads compose.yml / .env.example)	38 pass / 0 fail
bun run check:pgtypes	generated output current, query names unique

Still DRAFT / do not merge until ledger-v9 is the standard.

[acedward]

00029 (main @ a608fa6, PR #68) merged into ledger-v9 — 80bace3

git merge origin/main → 80bace37bc2412542452e1c597761b2ebce5c677, parents 358aaca (this branch, PR #67) + a608fa6 (main, PR #68). 31 files, no textual conflicts. Same precedent as 14782a2 (#63) and 4af1025 (#66); the reapply commit 544670d is still an ancestor, so the PR diff remains the whole ledger-v9 line.

What came in from main: blank/whitespace optional env now selects the package default (packages/price-feed/src/config.ts + packages/batcher/config.ts via package-local optionalString/optionalNumber), mint-test-tokens.ts registers TESTTOKENA/B/U through POST /v1/known-tokens behind a pure registerMintedTokenNames seam, and Compose plus both launchers use the linear contract → healthy kernel → mint/register/receipt → compatibility → wallet consumers startup (new mint-test-tokens one-shot and entrypoint-mint-test-tokens.sh).

Eight files were touched on both sides. Git auto-merged all eight; each was re-diffed against both parents and read line by line rather than trusted:

File	The risk	State after the merge
packages/contracts-midnight/mint-test-tokens.ts	#67 changed the mint_shielded/mint_unshielded signatures in place; #68 rewrote the registration block	Disjoint regions. v9's shieldedUserRecipient(walletResult.zswapSecretKeys.coinPublicKey) 4th argument and unshieldedUserRecipient(toHex(recipientBytes)) are intact; the old inline /api/known-tokens loop is gone. MINT_AMOUNT still uses coinsToBaseUnits(1000n, DEFAULT_TOKEN_DECIMALS)
start.dev.ts	#67 replaced the compact --version probe with the checksum-pinned infra/compact.sh route; #68 added the API-health edge	Both present: infra/compact-version.txt + execFileSync probe, and sync-api-health, ENABLE_TOKEN_REGISTRY: "true" on sync, ZSWAP_API on mint
deploy/compose.yml	#67 moved the indexer to api/v4 and disabled the proof-server healthcheck (PORT: "6300", consumers on service_started)	All six proof-server: service_started conditions and both api/v4 endpoint blocks survive; the mint-test-tokens service and ten service_completed_successfully edges are added on top
deploy/images/kernel/Dockerfile	#67 rewrote the compactc build stage and dropped the COMPACT_VERSION build arg	Only the COPY … entrypoint-mint-test-tokens.sh line and comment text came from main; v9's checksum-pinned compile stage is byte-identical
README.md, deploy/README.md, deploy/.env.example, deploy/scripts/provision-solver-fees.ts	—	Documentation/comment text only

No pin moves. bun.lock and every package.json are outside the merge diff. Post-merge: zero ledger-v8 in the lockfile, exactly one @midnightntwrk/ledger-v9 resolution (1.0.0-rc.3), exactly one @midnight-ntwrk/compact-runtime resolution (0.19.0), infra/compact-version.txt still 0.34.0.

Gates (space-free git clone --local of the merged tree at 80bace3, isolated auto-removed containers, bun install --frozen-lockfile → lockfile unchanged):

Gate	Image	Result
CI unit set (the exact list in ci.yml)	oven/bun:1.3.3	1596 pass / 2 skip / 1 fail, 17,140 expect() calls, 1,599 tests / 114 files
00029's own suites (packages/price-feed, register-known-tokens.test.ts, launcher-dependencies.test.ts, start.test.ts)	oven/bun:1.3.3	71 pass / 0 fail, 262 expect() calls
bun run typecheck	oven/bun:1.3.3	backend 0 / solver 0 / price-feed 0 diagnostics
bun run check:pgtypes	bun 1.3.3 + node 24 + git	generated output current, query names unique (11 query types)
bun run check:compact-runtime + bun test scripts/check-compact-runtime.test.ts	oven/bun:1.3.11	compact-runtime invariant OK: @midnight-ntwrk/compact-runtime@0.19.0; 4 pass / 0 fail

The single failure is host-dependent and named: packages/tests/grand-e2e/lib/solver-offerfiles-real-image-secret-scan.test.ts aborts in its beforeAll, which shells out to docker pull — error: Executable not found in $PATH: "docker" at line 42, so Bun reports the whole file as (fail) (unnamed). That file is untouched by this merge and by both parents, and CI's runner has Docker.

Not run here: check:compact-artifacts, deploy/gates.sh, the kernel image build and any Compose bring-up — all need image builds or containers on a Docker daemon another session owns, and this merge changes no dependency pin, no Dockerfile build stage and no Compact source. This PR's own compact-compile and kernel-image-amd64 jobs cover them.

Still DRAFT / do not merge until ledger-v9 is the standard.

[acedward]

main (#69) merged into ledger-v9

origin/main @ e3b9388d11dfe1a6c5554a4c8699250fe595e4ce (PR #69, Remove local faucet contract and use canonical external tokens) is now an ancestor of ledger-v9. This PR stays OPEN / DRAFT; nothing here was merged or retargeted.

Merge commit	2e0ba008ed245b9bb100e7753184e6fa8d722e06 — parents 560a77b79b0a718d00ac27d556a759e2763f0483 (ledger-v9, PR #70) + e3b9388d11dfe1a6c5554a4c8699250fe595e4ce (main, PR #69)
origin/ledger-v9 now	5d794f9a27f6d65529bf176650405f740531d430 — the PR #71 merge that carried 2e0ba00 onto the branch. Same tree
Method	merge commit only — no rebase, no cherry-pick, no reimplementation
git merge-base --is-ancestor 544670d HEAD	YES — the reapply commit is still an ancestor
git merge-base --is-ancestor origin/main HEAD	YES
git rev-list --left-right --count origin/main...HEAD	0 14 — behind 0, ahead 14
Merged tree	32076111f34238c19e3d0c43205c210cb19908c0

What came in from main

The four commits ledger-v9 was behind: e3b9388 (merge #69), 5e8d410 Remove local faucet contract and require prefunded inventory, f074494 Seed published Preprod test tokens, e439258 Externalize faucet and import canonical token registry.

In content: nothing. PR #70 is PR #69 already ported to the v9 line, and it turned out to be a complete port — the resolved tree is byte-identical to ledger-v9 @ 560a77b. This merge carries history, not content: it makes main an ancestor so the branch is no longer behind.

That is measured, not assumed:

• File coverage — git diff --name-status a608fa6 e3b9388 (Remove local faucet contract and use canonical external tokens #69) = 171 files, git diff --name-status 80bace3 560a77b (Remove local faucet contract on ledger-v9 #70) = 181 files, and comm of the two lists shows no file Remove local faucet contract and use canonical external tokens #69 touched that Remove local faucet contract on ledger-v9 #70 did not. The 10 extra are v9-only files (LEDGER-V9-MIGRATION.md, infra/compact-toolchain.Dockerfile, mint-recipient{,.test}.ts, PRODUCTION-READINESS.md, seed-market.ts, scripts/check-compact-{artifacts,runtime}.ts, check-compact-runtime.test.ts, start.external.ts).
• Line coverage — every line Remove local faucet contract and use canonical external tokens #69 adds (--unified=0, bun.lock excluded) searched verbatim in the merged tree: 4,464 added lines, 4,461 present, 3 absent, and those 3 are the two deliberate v9 differences: ["solver-provision"|"relay", "proof-server", "service_healthy"] (v9 uses service_started, because ⚠ BREAKING: move ledger-v9 to Compact 0.34 and typed mint recipients #67 disabled that healthcheck) and "@effectstream/wallets": "0.103.1" (v9 pins 0.200.2).

Conflicts — 11 files, all content conflicts, no delete/modify

Both sides deleted packages/contracts-midnight/** and git resolved those deletions silently. Rule applied: #70 IS #69 ported, so prefer the v9 side; take main's only where main has something v9 lacks — which, per the measurement above, was nowhere.

File	Resolution	Why
README.md	v9	v9's hunk is a superset — keeps the branch-pairing note (v-next↔ledger-v9, midnight-1↔main) and "Midnight 2.x / ledger-v9 only"; main's side of the hunk was empty. The canonical-token-registry paragraph both ports added is common text and survives
bun.lock (11 hunks)	v9	every hunk is a v9-vs-v8 pin block or a main-only transitive dedup entry for the v8 tree (onchain-runtime-v3, midnight-js-*@4.1.1, platform-js@2.2.4, wallet-sdk-address-format@3.1.0, bin-version/execa/*)
deploy/scripts/lib/check-compose-topology.ts	v9	proof-server edges stay service_started; main's service_healthy would fail against v9's own compose.yml. diff ours theirs shows those 3 lines are the only difference
deploy/scripts/lib/poster-config.test.ts	v9	both ports rewrote it (base 522 → v9 198 / main 168 lines) and v9 is a strict superset: it keeps "a malformed number or boolean is a startup error, never a silent default" and "network endpoints follow MIDNIGHT_NETWORK_ID, and explicit values win", which the main port deleted
docs/package.json	v9 pins	midnight-js-network-id 5.0.0-beta.6, @effectstream/wallets 0.200.2, mip-zswap-offer 0.4.0-v9.0
packages/database/migrations/000-init.sql	v9	the conflict is a single comment word (ledger-v9 vs ledger-v8); the seed rows are byte-identical on both sides
packages/midnight-infra/package.json	v9 pins	@effectstream/npm-midnight-* 0.200.2
packages/node/package.json	v9 pins	ledger-v9 1.0.0-rc.3 / onchain-runtime-v4 / midnight-js 5.0.0-beta.6. One non-pin difference checked: main lists @effectstream/event-client@0.103.1, v9 does not — git log -S shows the v9 line dropped that direct dependency in 544670d, long before either port; #69 never touched it and nothing imports it
packages/solver-core/package.json	v9 pins	+ mip-zswap-offer 0.4.0-v9.0
packages/tests/package.json	v9 pins	+ wallet-sdk-hd 3.1.0-beta.1, wallet-sdk-unshielded-wallet 4.0.0-beta.2
packages/tests/stm/unshielded-only.test.ts	v9	@midnightntwrk/… scope; v9 also carries signDataAsync and the waitForWalletSettlement error-170 guard that main's signData version lacks

Two of these needed whole-file --ours rather than hunk-level resolution, because git's 3-way output straddled the change boundary: check-compose-topology.ts would have ended up with a duplicated const REMOVED_SERVICES (both sides added the identical line at different offsets), and poster-config.test.ts lost two assertion lines from the middle of a test main deleted around.

Post-merge tree assertions

• ls packages/contracts-midnight → absent.
• grep -rn "mint_shielded\|mint_unshielded\|contract-offer-files" → 10 hits, all in LEDGER-V9-MIGRATION.md, under the Historical migration record banner Remove local faucet contract on ledger-v9 #70 added. No live code, script, Dockerfile or workflow reference remains.
• One 000-init.sql seed set, v9's: 8 known_tokens rows (NIGHT, SNIGHT, TWBTC, TWETH, TWUSDC, TWUSDM, UTWUSDC, UTWBTC) and 6 canonical_token_registry_state rows at revision ebd5eab….
• .github/workflows/ci.yml = v9's, a strict superset of main's — git diff origin/main HEAD -- .github/workflows has no removal lines at all. Adds scripts/check-compact-runtime.test.ts to unit-tests and the kernel-image-amd64 job (which asserts test ! -e /app/packages/contracts-midnight inside the built image).
• compactc is no longer a build input. Remove local faucet contract on ledger-v9 #70 deleted the whole compactc install stage from deploy/images/kernel/Dockerfile; root package.json has no check:compact-artifacts, scripts/check-compact-artifacts.ts is gone, ci.yml has no compact-compile job and no start.*.ts probes the compiler. infra/compact-version.txt (0.34.0), infra/compact-checksums.sha256, infra/compact-toolchain.Dockerfile and infra/compact.sh remain in the tree but are orphaned; the merge did not resurrect the stage from main. What is still live is the runtime pin only.

Gates

Space-free git clone --local copy, auto-removed docker run --rm containers, Docker 29.7.2, host arm64. No Compose stack started and no kernel image built — another executor owns this host's Docker.

Gate	Image	Result
bun install --frozen-lockfile	oven/bun:1.3.3 (CI's pin)	2,182 packages, bun.lock unchanged. The 3 failures are OPTIONAL native postinstalls (msgpackr-extract, bufferutil, utf-8-validate) with no node-gyp toolchain in the arm64 Bun image — CI's x64 runner uses prebuilds
lockfile pins	—	ledger-v8 0; one @midnightntwrk/ledger-v9@1.0.0-rc.3 resolution; one @midnight-ntwrk/compact-runtime resolution, 0.19.0 (4 transitive 0.18.0-rc.1 requests all dedupe onto it; installed version 0.19.0, no nested copy)
bun run typecheck	oven/bun:1.3.3	exit 0 — backend 0, solver 0, price-feed 0 in-scope diagnostics
CI unit set (exact ci.yml list)	oven/bun:1.3.3	1,462 pass / 1 fail / 0 skip, 14,027 assertions, 112 files
bun run check:compact-runtime + its test	oven/bun:1.3.11	exit 0 — compact-runtime invariant OK: @midnight-ntwrk/compact-runtime@0.19.0; 4 pass / 0 fail
bun run check:pgtypes	node:24-bookworm-slim + git + bun 1.3.3	exit 0 — 11 query types regenerated, generated output is current and query names are unique
non-CI suites (price-feed, launcher-dependencies, start)	oven/bun:1.3.3	65 pass / 0 fail

The one failure, named: packages/tests/grand-e2e/lib/solver-offerfiles-real-image-secret-scan.test.ts — its beforeAll runs docker pull and the test container has no Docker CLI (Executable not found in $PATH: "docker", line 42), so the file aborts in its hook and Bun reports (fail) (unnamed). Reproduced in isolation. Untouched by this merge and by both parents; green on GitHub's runner. Environment-only. No other failure exists.

Two pre-existing observations, neither caused by this merge (its tree is identical to 560a77b): bun run check:compact-runtime needs Bun ≥ 1.3.11 (Bun.JSONC.parse, introduced by d8d4ec8 in #67) while CI pins unit-tests to 1.3.3 — CI is unaffected because it runs only the test file, which imports the pure helpers; and packages/price-feed, packages/tests/launcher-dependencies.test.ts and packages/tests/start.test.ts are still outside the unit-tests list.

Not run, deliberately: deploy/gates.sh, the kernel image build and any Compose bring-up — this merge moves no pin, Dockerfile stage or Compose content, and this PR's own CI runs kernel-image-amd64.