Add dynamic overlay lookup discovery and bounded host scheduling - #518
Merged
Merged
Conversation
…duling Subscribe to SLAP tracker updates during an active query so later advertised hosts can join a fair, concurrency-bounded queue. Read lookup bodies incrementally with honest resource-limit accounting, keep trackers as routing sources, and leave raw query$ unverified while exposing the C02 onEvidence intake seam. Preserves 2s/5s delays, reputation/backoff, query/query$/freeform APIs, and existing CORS/public lookup request headers.
Preserve per-entry script results so a rejected sibling cannot poison a shared ancestor. LocalChainTracker tokens use participating sources only and fail closed on missing identity. Reset ownership is rechecked after dispose before destructive hooks. Built-in remote ChainTracks clients advertise reorg-event capability explicitly. Services.getChainTracker publishes one coalesced wrapper before yielding.
BHServiceClient no longer caches the queried merkle root. Validity is decided from a freshly read header, so a false result cannot invert on retry and a reorg cannot reuse a stale positive.
Store discovery bounds on the SLAP host cache so a tighter-limit query cannot freeze a truncated host set for a later broader attempt. Throw the historical no-competent-hosts error from query()/queryDetailed when a deadline expires before any host is admitted; query$ still emits the deadline snapshot.
LookupResolver is a BRC-100 JSON boundary file, so the in-flight discovery key cannot use JSON.stringify. stringifyBRC100 preserves the same service-plus-limit tuple without changing cache cover semantics.
Drop redundant optional-undefined on lookup evidence limits, extract parseEvidence and Chaintracks header-retry helpers below S3776, and rename coordinator catch params to error_.
…e-c02 # Conflicts: # docs/reference/package-api-migrations.md # governance/package-release-notes.json
Extract query/session and host-tracking helpers so cognitive complexity stays within Sonar's limit, and clear the remaining overlay-tools findings without changing lookup behavior.
S7746 on LookupResolver: prefer throw over return Promise.reject in the facilitator start chain.
discoverOverlayCertificates called getServices(), which throws on wallets constructed without services and broke BRC-100 discoverBy* conformance. Missing chain context now yields no identities instead of throwing.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Health requires overrideRemovalReview.retainedCount to match the live registry.
Health requires overrideRemovalReview.retainedCount to match the live registry.
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates.
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates.
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth.
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth.
…ry-c04 # Conflicts: # packages/messaging/message-box-client/browser-budget.json # packages/sdk/browser-budget.json
Merge origin/main (bdaebe6) into prep/517 and resolve the metadata drift this branch carried. - Restore pnpm-workspace.yaml, governance/repository-health/exceptions.json, governance/dependency-release-policy.json and pnpm-lock.yaml to main's copies. Main already ships the js-yaml 3.15.2 override and registers it as js-yaml-jest-istanbul-advisory-override, so this branch's relocation and its duplicate js-yaml-jest-advisory-override entry were redundant and produced a duplicated overrides key in the merged lockfile (ERR_PNPM_BROKEN_LOCKFILE). - Raise the bundle and platform budgets to the wave ceilings, which an earlier main merge on this branch had silently reverted. - Drop the stray .claude/tasks/context_session_c02.md working note. - Regenerate the ts2md reference docs for the overlay-tools and transaction modules so they match the source in this branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Resolve conflicts in browser/platform budget JSONs (take the field-wise maximum ceilings) and pnpm-lock.yaml (take main's lockfile; no new dependencies introduced by this PR). Restore scripts/dependency-release-governance.test.mjs to main's copy, which already contains the single assertion this PR duplicated. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Merge #517's onEvidence/evidenceLimits evidence-intake seam onto #518's bounded discovery/scheduling architecture in LookupResolver.ts. Keep #518's LookupDiscovery/LookupHostQueue/LookupResources rewrite as the base; it already delivers owned per-host output copies to onEvidence before legacy txid/outpoint dedup, caps evidence intake at 512 outputs / 16 MiB by default, and emits exactly one {type:'limit'} event per query. Restore the exact "Evidence intake limits must be positive safe integers" validation message for the evidenceLimits shorthand (previously only in #517's LookupQuerySession constructor) by validating it in a new lookupQueryLimits/evidenceLimitOverrides helper before any host is queried, and document the precedence between evidenceLimits and limits.maxEvidenceOutputs/maxEvidenceBytes when both are supplied for the same call. Keep a single formatting choice for the now-identical LookupEvidenceEvent union. Regenerate packages/sdk/docs/reference/overlay-tools.md (and the rest of the ts2md-managed SDK reference docs, which had drifted out of sync with source) via `pnpm --filter @bsv/sdk run doc` instead of hand merging. No version bumps. No behavior from either PR's test suite is removed or loosened. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ling The main integration merge raised the SDK esbuild raw ceiling to 600000, a value that belongs to the lookup discovery change (#518). This branch measured 590000; restore it so the diff carries only its own budget. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Restore the eleven ts2md reference pages for modules this change does not touch to the incoming copies; the generator run had imported unrelated drift that already exists on main. Keep the regenerated overlay-tools page. Reinstate this change's own 600000 esbuild raw ceiling after restacking on the corrected evidence branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
query() and queryDetailed() flatten query$ to its first snapshot. The
terminal gate only threw when hostCount was 0, and explicitly skipped
the throw for terminalReason 'cancelled', so a caller abort resolved as
{ type: 'output-list', outputs: [] }. At any host count an aborted
attempt was indistinguishable from an authoritative "this service has
no outputs" answer, which is exactly the completion-vs-empty confusion
queryDetailed exists to prevent.
Throw lookupAbortError() before the host-count gate whenever the
terminal snapshot reports 'cancelled'. query$ is unchanged and still
emits its terminalReason: 'cancelled' snapshot for progressive callers.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
chargeLookupDiscoveryBytes can exhaust maxTotalBytes while SLAP trackers are still being read, before any host is admitted. That leaves the terminal snapshot with hostCount 0, terminalReason 'resource-limit' and the limit recorded in limitsHit. queryDetailed()'s hostCount gate did not look at terminalReason, so it threw the historical "No competent <network> hosts found by the SLAP trackers" Error and discarded both fields. Security-sensitive Promise callers could not tell "the trackers named no hosts" from "this attempt ran out of its own byte budget". Branch the gate on terminalReason: 'resource-limit' throws LookupResourceLimitError carrying the first limit that was hit, while 'deadline' and a settled empty discovery keep the historical error and message unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
performLookupRequest built its RequestInit without a redirect field, so
fetch defaulted to 'follow'. normalizeLookupHost and the https: guard in
lookup() validate the advertised URL only; neither runs again on the
final URL. A SLAP-advertised HTTPS host could therefore answer the
lookup POST with a 307/308 and have the serialized
stringifyBRC100({ service, query }) body re-sent to http:, loopback or a
link-local address. This PR queries every eligible advertised host, so
the hop sat on the hot path for untrusted SLAP domains, and the SLAP
tracker discovery requests reach the network through this same method.
Set redirect: 'error' so the transport fails closed. fetch rejects, and
the existing failure path records the advertised host as an ordinary
availability failure instead of crashing the query or following the hop.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… them
TransactionEvidenceCoordinator.run() shifts the next candidate off a job's
queue before calling attempt(). If every concurrency slot is already in use
(including slots still reserved by non-abortable, already-finished attempts;
see the "does not free a non-abortable backend slot..." test), attempt()
threw a plain TransactionEvidenceError('limit') and called the caller's
`settled` callback. run()'s catch treated that exactly like a genuine
verification failure: the candidate was gone (already shift()ed) and never
pushed back, so a same-txid alternate candidate that had already been
admitted to the job could be silently discarded without ever being tried,
even though it was perfectly valid.
Fix: attempt() now throws a dedicated ConcurrencyLimitSignal when the
operation never started, and does not invoke `settled` (nothing was
consumed, so no byte/candidate bookkeeping should be released). run()'s
candidate loop catches that signal specifically, unshifts the candidate
back onto job.candidates, marks the job not-running, and returns without
looping or retrying itself. Re-admission is driven solely by pump() being
invoked again when some other active attempt settles, so this cannot spin
or busy-wait for a slot.
Added a regression test that builds a genuine concurrency race: a
non-abortable "ghost" attempt (cancelled while its script verification is
still in flight) holds one of two slots, a job with two same-txid
candidates (one invalid, one valid) takes the other, and a third waiting
job is admitted into the slot freed by the first candidate's failure before
the job's own retry can reclaim it. Before the fix this made the valid
alternate candidate reject with 'limit'; the test fails for that reason on
the old code and passes with the fix.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every TransactionEvidenceCoordinator attempt calls
ChaintracksChainTracker.getVerificationContextToken(), which requires
chaintracks.findChainTipHash(). BHServiceClient implemented that method
as `throw new Error('Not implemented')` even though it already exposes a
working findChainTipHeader() against /api/v1/chain/tip/longest. Any wallet
using a BHServiceClient as options.chaintracks, or as a LocalChainTracker
participating source, broke on every verification attempt.
Fix: implement findChainTipHash() by delegating to the existing
findChainTipHeader() and returning its hash, matching the same pattern
GoChaintracksServiceClient already uses for the same interface method.
No other ChaintracksClientApi contract surface changes.
Added a regression test on BHServiceClient confirming findChainTipHash()
resolves to findChainTipHeader()'s hash instead of throwing, and a
regression test on ChaintracksChainTracker confirming
getVerificationContextToken() succeeds end-to-end when backed by a
BHServiceClient. Both fail with "Not implemented" on the old code and
pass with the fix.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Document the TransactionEvidenceCoordinator candidate-requeue fix and the BHServiceClient.findChainTipHash implementation in their packages' CHANGELOG.md, matching this repo's convention of recording behavior- affecting fixes there alongside the commit history. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The concurrency-limit signal is handled only in the candidate loop. The cached-positive recheck does not need the same handling: run() is reached solely from pump(), which checks the attempt limit synchronously before starting the job, and no await separates that check from the recheck attempt. Record the invariant next to the call so a later refactor of pump() or run() revisits it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dles The ceilings carried on this branch were authored before main grew (#534, #547), so the merged wallet bundles exceeded them and CI stopped at the first over-budget dimension. Measured from the exact packed dependency graph at 83c0670 (JS bundle sizes verified byte-identical between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio recorded in #547): client Vite 1,759,717 raw / 416,284 gzip / 324,287 brotli client esbuild 1,372,320 raw / 378,741 gzip / 302,937 brotli mobile Metro 1,811,581 raw / 461,355 gzip / 355,871 brotli mobile Hermes 3,688,871 raw / 1,502,972 gzip / 1,166,821 brotli Only the dimensions that exceeded are raised, with about 0.25% headroom (1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Bounded discovery, the host queue and the streaming response reader grow every bundle that embeds @bsv/sdk. Measured from the exact packed dependency graph at the top of the ordered overlay wave (max of 5eef0fc and 9077394; JS bundle sizes verified byte-identical between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio from #547): @bsv/sdk UMD 591,705 raw / 168,531 gzip / 140,485 brotli @bsv/simple Vite 752,122 raw / 182,787 gzip / 150,699 brotli wallet client Vite 1,782,018 raw / 421,917 gzip / 328,724 brotli wallet client esbuild 1,389,928 raw / 383,620 gzip / 306,783 brotli wallet mobile Metro 1,834,234 raw / 466,804 gzip / 360,591 brotli wallet mobile Hermes 3,725,120 raw / 1,520,644 gzip / 1,181,384 brotli Only the dimensions that exceeded are raised, with about 0.25% headroom (1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance). Later branches in the wave move these bundles by under 150 bytes, so they inherit these ceilings unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Cover the cancellation and resource-limit branches of HTTPSOverlayLookupFacilitator: a pre-aborted caller signal short-circuits before any fetch, an abort racing a non-ok response reports cancellation and drains the body, JSON and octet-stream output counts are rejected above the caller's budget (including a negative varint count), atomic BEEF extraction stops once the extracted bytes outrun the response budget, and an in-flight decode observes a mid-stream cancellation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…unds Exercise the uncovered fail-closed branches of LookupResolver.query$: aggregation stopping at maxOutputs across hosts, the evidenceLimits shorthand defaulting its byte budget, the bounded candidate scan (scan window, malformed entries, maxHosts), per-source quotas for additional hosts, SLAP discovery refusing bytes that would breach maxTotalBytes, byte reports arriving after cancellation, cancellation dropping an in-flight peer answer and skipping queued hosts, idempotent double cancellation, the concurrent-query ceiling, over-budget answers being dropped without blaming the host, the maxTrackers budget, advertisement-map eviction, and a broader host cache surviving a tighter rediscovery. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
sirdeggen
added a commit
that referenced
this pull request
Sep 19, 2026
* fix(wallet-toolbox): verify overlay identity evidence * fix(sdk): close C02/C03 evidence coordinator review findings Preserve per-entry script results so a rejected sibling cannot poison a shared ancestor. LocalChainTracker tokens use participating sources only and fail closed on missing identity. Reset ownership is rechecked after dispose before destructive hooks. Built-in remote ChainTracks clients advertise reorg-event capability explicitly. Services.getChainTracker publishes one coalesced wrapper before yielding. * fix(wallet-toolbox): re-read BHS canonical headers for root checks BHServiceClient no longer caches the queried merkle root. Validity is decided from a freshly read header, so a false result cannot invert on retry and a reorg cannot reuse a stale positive. * fix(sonar): evidence coordinator and chaintracks findings Drop redundant optional-undefined on lookup evidence limits, extract parseEvidence and Chaintracks header-retry helpers below S3776, and rename coordinator catch params to error_. * fix(wallet-toolbox): skip overlay identity when services are absent discoverOverlayCertificates called getServices(), which throws on wallets constructed without services and broke BRC-100 discoverBy* conformance. Missing chain context now yields no identities instead of throwing. * fix(ci): override js-yaml 3.15.2 for GHSA-2883 Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high. * fix(ci): record js-yaml override removal rehearsal Health requires overrideRemovalReview.retainedCount to match the live registry. * fix(ci): ratchet override count for js-yaml 3.15.2 * fix(ci): raise SDK and wallet platform bundle budgets LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates. * fix(ci): raise remaining SDK-consumer bundle budgets Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth. * fix(ci): raise wallet client esbuild brotli budget * fix(ci): raise did-client UMD budget for SDK helper growth * fix(ci): prettier browser budget JSON * fix(ci): raise did-client esbuild budget * fix(ci): raise did-client vite budget * fix(ci): raise SDK vite and esbuild compressed budgets * fix(wallet-toolbox): throw on overlay identity forceRefresh without services Contact discovery still works without services. forceRefresh bypasses contacts and still requires a chain tracker, matching the existing test. * fix(sonar): extract overlay chain-tracker guard from Wallet discovery S3776: discoverOverlayCertificates was 17 after the forceRefresh services check. The missing-services path is now requireOverlayChainTracker. * test(sdk): cover chain tracker and evidence helpers for patch coverage * fix(ci): parent and finalize Codecov coverage reports * docs: reverify resource and TTN rollout gates * fix(ci): require HTTPS for Codecov polling * test(sdk): cover invalid evidence intake limits * fix(ci): remove duplicate workspace override * fix: avoid eager fetch dependency in broadcaster * fix: defer unavailable fetch failure until send * fix(sdk): keep the evidence branch esbuild budget at its measured ceiling The main integration merge raised the SDK esbuild raw ceiling to 600000, a value that belongs to the lookup discovery change (#518). This branch measured 590000; restore it so the diff carries only its own budget. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): re-queue displaced same-txid candidates instead of dropping them TransactionEvidenceCoordinator.run() shifts the next candidate off a job's queue before calling attempt(). If every concurrency slot is already in use (including slots still reserved by non-abortable, already-finished attempts; see the "does not free a non-abortable backend slot..." test), attempt() threw a plain TransactionEvidenceError('limit') and called the caller's `settled` callback. run()'s catch treated that exactly like a genuine verification failure: the candidate was gone (already shift()ed) and never pushed back, so a same-txid alternate candidate that had already been admitted to the job could be silently discarded without ever being tried, even though it was perfectly valid. Fix: attempt() now throws a dedicated ConcurrencyLimitSignal when the operation never started, and does not invoke `settled` (nothing was consumed, so no byte/candidate bookkeeping should be released). run()'s candidate loop catches that signal specifically, unshifts the candidate back onto job.candidates, marks the job not-running, and returns without looping or retrying itself. Re-admission is driven solely by pump() being invoked again when some other active attempt settles, so this cannot spin or busy-wait for a slot. Added a regression test that builds a genuine concurrency race: a non-abortable "ghost" attempt (cancelled while its script verification is still in flight) holds one of two slots, a job with two same-txid candidates (one invalid, one valid) takes the other, and a third waiting job is admitted into the slot freed by the first candidate's failure before the job's own retry can reclaim it. Before the fix this made the valid alternate candidate reject with 'limit'; the test fails for that reason on the old code and passes with the fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(wallet-toolbox): implement BHServiceClient.findChainTipHash Every TransactionEvidenceCoordinator attempt calls ChaintracksChainTracker.getVerificationContextToken(), which requires chaintracks.findChainTipHash(). BHServiceClient implemented that method as `throw new Error('Not implemented')` even though it already exposes a working findChainTipHeader() against /api/v1/chain/tip/longest. Any wallet using a BHServiceClient as options.chaintracks, or as a LocalChainTracker participating source, broke on every verification attempt. Fix: implement findChainTipHash() by delegating to the existing findChainTipHeader() and returning its hash, matching the same pattern GoChaintracksServiceClient already uses for the same interface method. No other ChaintracksClientApi contract surface changes. Added a regression test on BHServiceClient confirming findChainTipHash() resolves to findChainTipHeader()'s hash instead of throwing, and a regression test on ChaintracksChainTracker confirming getVerificationContextToken() succeeds end-to-end when backed by a BHServiceClient. Both fail with "Not implemented" on the old code and pass with the fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: changelog entries for F1/F2 review-finding fixes Document the TransactionEvidenceCoordinator candidate-requeue fix and the BHServiceClient.findChainTipHash implementation in their packages' CHANGELOG.md, matching this repo's convention of recording behavior- affecting fixes there alongside the commit history. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(sdk): state why a cached evidence recheck cannot be displaced The concurrency-limit signal is handled only in the candidate loop. The cached-positive recheck does not need the same handling: run() is reached solely from pump(), which checks the attempt limit synchronously before starting the job, and no await separates that check from the recheck attempt. Record the invariant next to the call so a later refactor of pump() or run() revisits it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(wallet): set platform budgets from measured evidence-branch bundles The ceilings carried on this branch were authored before main grew (#534, #547), so the merged wallet bundles exceeded them and CI stopped at the first over-budget dimension. Measured from the exact packed dependency graph at 83c0670 (JS bundle sizes verified byte-identical between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio recorded in #547): client Vite 1,759,717 raw / 416,284 gzip / 324,287 brotli client esbuild 1,372,320 raw / 378,741 gzip / 302,937 brotli mobile Metro 1,811,581 raw / 461,355 gzip / 355,871 brotli mobile Hermes 3,688,871 raw / 1,502,972 gzip / 1,166,821 brotli Only the dimensions that exceeded are raised, with about 0.25% headroom (1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> # Conflicts: # packages/sdk/browser-budget.json # packages/sdk/docs/reference/overlay-tools.md # packages/sdk/src/overlay-tools/LookupResolver.ts # packages/wallet/wallet-toolbox/client/platform-budget.json # packages/wallet/wallet-toolbox/mobile/platform-budget.json
|
sirdeggen
added a commit
that referenced
this pull request
Sep 19, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
sirdeggen
added a commit
that referenced
this pull request
Sep 19, 2026
* fix(wallet-toolbox): verify overlay identity evidence * feat(overlay): define optional admission and recovery contracts * feat(sdk): add dynamic overlay lookup discovery and bounded host scheduling Subscribe to SLAP tracker updates during an active query so later advertised hosts can join a fair, concurrency-bounded queue. Read lookup bodies incrementally with honest resource-limit accounting, keep trackers as routing sources, and leave raw query$ unverified while exposing the C02 onEvidence intake seam. Preserves 2s/5s delays, reputation/backoff, query/query$/freeform APIs, and existing CORS/public lookup request headers. * feat(overlay): add opt-in Mongo payload and schema foundation Add replica-set schema bootstrap, content-addressed payload publication, transactionally guarded references/GC, and majority-commit outcome helpers behind an optional mongodb peer. This is not an AdmissionStorage adapter and does not change Engine or Knex defaults. * fix(sdk): close C02/C03 evidence coordinator review findings Preserve per-entry script results so a rejected sibling cannot poison a shared ancestor. LocalChainTracker tokens use participating sources only and fail closed on missing identity. Reset ownership is rechecked after dispose before destructive hooks. Built-in remote ChainTracks clients advertise reorg-event capability explicitly. Services.getChainTracker publishes one coalesced wrapper before yielding. * fix(wallet-toolbox): re-read BHS canonical headers for root checks BHServiceClient no longer caches the queried merkle root. Validity is decided from a freshly read header, so a false result cannot invert on retry and a reorg cannot reuse a stale positive. * fix(sdk): keep overlay host cache and query() deadline honest Store discovery bounds on the SLAP host cache so a tighter-limit query cannot freeze a truncated host set for a later broader attempt. Throw the historical no-competent-hosts error from query()/queryDetailed when a deadline expires before any host is admitted; query$ still emits the deadline snapshot. * fix(overlay): compare Mongo uint32 indexes as integers Collection validators used lexicographic $lte on unpadded outputIndex strings, which rejected legal indexes such as "9". Bound uint32 fields with $toLong on outputs, consumption edges, GASP nodes, and SHIP/SLAP. * fix(sdk): encode overlay discovery cache keys with stringifyBRC100 LookupResolver is a BRC-100 JSON boundary file, so the in-flight discovery key cannot use JSON.stringify. stringifyBRC100 preserves the same service-plus-limit tuple without changing cache cover semantics. * fix(sonar): evidence coordinator and chaintracks findings Drop redundant optional-undefined on lookup evidence limits, extract parseEvidence and Chaintracks header-retry helpers below S3776, and rename coordinator catch params to error_. * fix(sonar): overlay lookup discovery complexity and style Extract query/session and host-tracking helpers so cognitive complexity stays within Sonar's limit, and clear the remaining overlay-tools findings without changing lookup behavior. * fix(sonar): overlay mongo and admission findings Brand StorageUint64, extract high-complexity helpers, and apply Sonar-preferred optional chaining, regex, clone, and sort APIs without changing payload identity or uint32 $toLong validators. * fix(sonar): throw lookup abort instead of Promise.reject S7746 on LookupResolver: prefer throw over return Promise.reject in the facilitator start chain. * fix(wallet-toolbox): skip overlay identity when services are absent discoverOverlayCertificates called getServices(), which throws on wallets constructed without services and broke BRC-100 discoverBy* conformance. Missing chain context now yields no identities instead of throwing. * fix(ci): override js-yaml 3.15.2 for GHSA-2883 Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high. * fix(ci): override js-yaml 3.15.2 for GHSA-2883 Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high. * fix(ci): align sdk 2.5.0 health baseline and stack facts Merge from main left baselines.json on 2.4.2 while package.json is 2.5.0. * fix(ci): override js-yaml 3.15.2 for GHSA-2883 Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high. * fix(ci): record js-yaml override removal rehearsal Health requires overrideRemovalReview.retainedCount to match the live registry. * fix(ci): record js-yaml override removal rehearsal Health requires overrideRemovalReview.retainedCount to match the live registry. * fix(ci): record js-yaml override removal rehearsal Health requires overrideRemovalReview.retainedCount to match the live registry. * fix(ci): ratchet override count for js-yaml 3.15.2 * fix(ci): ratchet override count for js-yaml 3.15.2 * fix(ci): ratchet override count for js-yaml 3.15.2 * fix(ci): raise SDK and wallet platform bundle budgets LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates. * fix(ci): raise SDK and wallet platform bundle budgets LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates. * fix(ci): raise SDK and wallet platform bundle budgets LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates. * fix(ci): raise remaining SDK-consumer bundle budgets Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth. * fix(ci): raise remaining SDK-consumer bundle budgets Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth. * fix(ci): raise remaining SDK-consumer bundle budgets Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth. * fix(ci): raise wallet client esbuild brotli budget * fix(ci): raise wallet client esbuild brotli budget * fix(ci): raise wallet client esbuild brotli budget * fix(ci): raise did-client UMD budget for SDK helper growth * fix(ci): raise did-client UMD budget for SDK helper growth * fix(ci): raise did-client UMD budget for SDK helper growth * fix(ci): prettier browser budget JSON * fix(ci): prettier browser budget JSON * fix(ci): prettier browser budget JSON * fix(ci): raise did-client esbuild budget * fix(ci): raise did-client esbuild budget * fix(ci): raise did-client esbuild budget * fix(ci): raise did-client vite budget * fix(ci): raise did-client vite budget * fix(ci): raise did-client vite budget * fix(ci): raise SDK vite and esbuild compressed budgets * fix(ci): raise SDK vite and esbuild compressed budgets * fix(ci): raise SDK vite and esbuild compressed budgets * fix(wallet-toolbox): throw on overlay identity forceRefresh without services Contact discovery still works without services. forceRefresh bypasses contacts and still requires a chain tracker, matching the existing test. * fix(sonar): extract overlay chain-tracker guard from Wallet discovery S3776: discoverOverlayCertificates was 17 after the forceRefresh services check. The missing-services path is now requireOverlayChainTracker. * test(sdk): cover overlay discovery helpers for patch coverage * test(overlay): cover mongo storage helpers for patch coverage * fix(sonar): use toHaveLength in LookupResolver discovery tests * test(sdk): cover chain tracker and evidence helpers for patch coverage * fix(ci): parent and finalize Codecov coverage reports * fix(ci): parent and finalize Codecov coverage reports * docs: reverify resource and TTN rollout gates * docs: reverify resource and TTN rollout gates * fix(ci): require HTTPS for Codecov polling * fix(ci): require HTTPS for Codecov polling * fix(ci): raise SDK esbuild raw browser budget * test(sdk): cover invalid evidence intake limits * test(overlay): cover uint64 payload length boundary * fix(ci): remove duplicate workspace override * fix: make overlay admission runtime portable * fix: avoid eager fetch dependency in broadcaster * fix: defer unavailable fetch failure until send * fix: defer unavailable fetch failure until send * chore(overlays): re-derive release metadata after the ordered merges Refreshes the overlay package documentation dates for the four packages this change bumps and regenerates the derived reference documents so the stack facts and package API migration tables match the merged versions. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): keep the evidence branch esbuild budget at its measured ceiling The main integration merge raised the SDK esbuild raw ceiling to 600000, a value that belongs to the lookup discovery change (#518). This branch measured 590000; restore it so the diff carries only its own budget. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): scope lookup discovery docs and budget to this change Restore the eleven ts2md reference pages for modules this change does not touch to the incoming copies; the generator run had imported unrelated drift that already exists on main. Keep the regenerated overlay-tools page. Reinstate this change's own 600000 esbuild raw ceiling after restacking on the corrected evidence branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): reject cancelled lookups instead of answering empty query() and queryDetailed() flatten query$ to its first snapshot. The terminal gate only threw when hostCount was 0, and explicitly skipped the throw for terminalReason 'cancelled', so a caller abort resolved as { type: 'output-list', outputs: [] }. At any host count an aborted attempt was indistinguishable from an authoritative "this service has no outputs" answer, which is exactly the completion-vs-empty confusion queryDetailed exists to prevent. Throw lookupAbortError() before the host-count gate whenever the terminal snapshot reports 'cancelled'. query$ is unchanged and still emits its terminalReason: 'cancelled' snapshot for progressive callers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): keep discovery resource limits out of the no-hosts error chargeLookupDiscoveryBytes can exhaust maxTotalBytes while SLAP trackers are still being read, before any host is admitted. That leaves the terminal snapshot with hostCount 0, terminalReason 'resource-limit' and the limit recorded in limitsHit. queryDetailed()'s hostCount gate did not look at terminalReason, so it threw the historical "No competent <network> hosts found by the SLAP trackers" Error and discarded both fields. Security-sensitive Promise callers could not tell "the trackers named no hosts" from "this attempt ran out of its own byte budget". Branch the gate on terminalReason: 'resource-limit' throws LookupResourceLimitError carrying the first limit that was hit, while 'deadline' and a settled empty discovery keep the historical error and message unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): refuse redirects on overlay lookup and SLAP discovery requests performLookupRequest built its RequestInit without a redirect field, so fetch defaulted to 'follow'. normalizeLookupHost and the https: guard in lookup() validate the advertised URL only; neither runs again on the final URL. A SLAP-advertised HTTPS host could therefore answer the lookup POST with a 307/308 and have the serialized stringifyBRC100({ service, query }) body re-sent to http:, loopback or a link-local address. This PR queries every eligible advertised host, so the hop sat on the hot path for untrusted SLAP domains, and the SLAP tracker discovery requests reach the network through this same method. Set redirect: 'error' so the transport fails closed. fetch rejects, and the existing failure path records the advertised host as an ordinary availability failure instead of crashing the query or following the hop. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(overlay-mongo): serialize pin re-add expiry with GC reclaim addReference's idempotent re-add branch only re-validated the payload was still ready; it never touched the existing (ownerKind, ownerId, slot) pin row's expiresAt. Extending a pin therefore silently kept the stale expiry, and claimGarbage counted an already-expired pin as not-live via $expr {$gt:['$expiresAt','$$NOW']} but never deleted that expired pin document. The unique slot row survived expiry and kept short-circuiting future addReference calls on that slot, so a caller could be told content stayed pinned while GC moved the payload ready -> deleting underneath it. Add refreshPinExpiry: called after the existing {_id: payloadId, state: 'ready'} CAS matches (so it stays serialized against a concurrent GC claim in the same transaction), it reactivates an expired pin with the caller's new expiry, allows extending a still- live pin to a later expiry, and rejects an attempt to shorten a live pin's expiry rather than silently ignoring it. claimGarbage now deletes every expired pin reference row for the payload in the same transaction as its ready -> deleting claim, so a stale slot cannot outlive the payload it named. Document the re-add/extend/reactivate/shorten rule in specs/overlay/mongo-v1.md next to the existing pin semantics. Regression tests (mongodb-memory-server replica fixture) added to src/__tests/mongo/MongoPayloadStore.test.ts: - re-adding a pin slot extends its expiry but never shortens it silently - re-adding an expired pin slot reactivates it and makes the payload live again - claimGarbage deletes the expired pin reference row inside the same transaction as its claim All three were confirmed to fail against the pre-fix source for the reason described above (verified by temporarily reverting this file and re-running the suite) before the fix was applied. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay-mongo): cover the too-small declared length reclaim path The reviewer asked for a regression test on the reclaim path taken when publish fails because the caller declared a too-small byteLength: consumePayloadChunk rejects the stream once it exceeds the declared length, abandonUpload marks the payload row 'deleted', and a retry with the correct digest and length must reclaim that row through tryClaimUpload (which already recomputes byteLength from the new input when the prior state is 'deleted') and finish 'ready' with the corrected byteLength stored. This path was already implemented and covered indirectly (a sibling test manually sets state: 'deleted' with a wrong byteLength via direct DB mutation), but there was no test driving the actual publish/fail/ retry sequence end to end. This is coverage-only: the reclaim logic in tryClaimUpload predates this change and needed no fix, so there is no meaningful pre-fix state to fail against in this worktree; the new test passes against the current code, which is the expected outcome for confirming existing, correct behavior. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): re-queue displaced same-txid candidates instead of dropping them TransactionEvidenceCoordinator.run() shifts the next candidate off a job's queue before calling attempt(). If every concurrency slot is already in use (including slots still reserved by non-abortable, already-finished attempts; see the "does not free a non-abortable backend slot..." test), attempt() threw a plain TransactionEvidenceError('limit') and called the caller's `settled` callback. run()'s catch treated that exactly like a genuine verification failure: the candidate was gone (already shift()ed) and never pushed back, so a same-txid alternate candidate that had already been admitted to the job could be silently discarded without ever being tried, even though it was perfectly valid. Fix: attempt() now throws a dedicated ConcurrencyLimitSignal when the operation never started, and does not invoke `settled` (nothing was consumed, so no byte/candidate bookkeeping should be released). run()'s candidate loop catches that signal specifically, unshifts the candidate back onto job.candidates, marks the job not-running, and returns without looping or retrying itself. Re-admission is driven solely by pump() being invoked again when some other active attempt settles, so this cannot spin or busy-wait for a slot. Added a regression test that builds a genuine concurrency race: a non-abortable "ghost" attempt (cancelled while its script verification is still in flight) holds one of two slots, a job with two same-txid candidates (one invalid, one valid) takes the other, and a third waiting job is admitted into the slot freed by the first candidate's failure before the job's own retry can reclaim it. Before the fix this made the valid alternate candidate reject with 'limit'; the test fails for that reason on the old code and passes with the fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(wallet-toolbox): implement BHServiceClient.findChainTipHash Every TransactionEvidenceCoordinator attempt calls ChaintracksChainTracker.getVerificationContextToken(), which requires chaintracks.findChainTipHash(). BHServiceClient implemented that method as `throw new Error('Not implemented')` even though it already exposes a working findChainTipHeader() against /api/v1/chain/tip/longest. Any wallet using a BHServiceClient as options.chaintracks, or as a LocalChainTracker participating source, broke on every verification attempt. Fix: implement findChainTipHash() by delegating to the existing findChainTipHeader() and returning its hash, matching the same pattern GoChaintracksServiceClient already uses for the same interface method. No other ChaintracksClientApi contract surface changes. Added a regression test on BHServiceClient confirming findChainTipHash() resolves to findChainTipHeader()'s hash instead of throwing, and a regression test on ChaintracksChainTracker confirming getVerificationContextToken() succeeds end-to-end when backed by a BHServiceClient. Both fail with "Not implemented" on the old code and pass with the fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: changelog entries for F1/F2 review-finding fixes Document the TransactionEvidenceCoordinator candidate-requeue fix and the BHServiceClient.findChainTipHash implementation in their packages' CHANGELOG.md, matching this repo's convention of recording behavior- affecting fixes there alongside the commit history. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(sdk): state why a cached evidence recheck cannot be displaced The concurrency-limit signal is handled only in the candidate loop. The cached-positive recheck does not need the same handling: run() is reached solely from pump(), which checks the attempt limit synchronously before starting the job, and no await separates that check from the recheck attempt. Record the invariant next to the call so a later refactor of pump() or run() revisits it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(wallet): set platform budgets from measured evidence-branch bundles The ceilings carried on this branch were authored before main grew (#534, #547), so the merged wallet bundles exceeded them and CI stopped at the first over-budget dimension. Measured from the exact packed dependency graph at 83c0670 (JS bundle sizes verified byte-identical between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio recorded in #547): client Vite 1,759,717 raw / 416,284 gzip / 324,287 brotli client esbuild 1,372,320 raw / 378,741 gzip / 302,937 brotli mobile Metro 1,811,581 raw / 461,355 gzip / 355,871 brotli mobile Hermes 3,688,871 raw / 1,502,972 gzip / 1,166,821 brotli Only the dimensions that exceeded are raised, with about 0.25% headroom (1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(budgets): set ceilings from measured lookup discovery bundles Bounded discovery, the host queue and the streaming response reader grow every bundle that embeds @bsv/sdk. Measured from the exact packed dependency graph at the top of the ordered overlay wave (max of 5eef0fc and 9077394; JS bundle sizes verified byte-identical between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio from #547): @bsv/sdk UMD 591,705 raw / 168,531 gzip / 140,485 brotli @bsv/simple Vite 752,122 raw / 182,787 gzip / 150,699 brotli wallet client Vite 1,782,018 raw / 421,917 gzip / 328,724 brotli wallet client esbuild 1,389,928 raw / 383,620 gzip / 306,783 brotli wallet mobile Metro 1,834,234 raw / 466,804 gzip / 360,591 brotli wallet mobile Hermes 3,725,120 raw / 1,520,644 gzip / 1,181,384 brotli Only the dimensions that exceeded are raised, with about 0.25% headroom (1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance). Later branches in the wave move these bundles by under 150 bytes, so they inherit these ceilings unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): exclude src/__tests support modules from coverage The repository's patch-coverage gate (scripts/patch-coverage.mjs, TEST_PATH pattern /(?:^|\/)__tests(?:__)?(?:\/|$)/) treats both `__tests/` and `__tests__/` directories as test code. This package's jest collectCoverageFrom only excluded `__tests__/**`, so the non-`.test.ts` support modules this PR adds under src/__tests/ (admission/ReferenceAdmissionStorage.ts, admission/AdmissionStorageContract.ts, mongo/MongoCommitResponseProxy.ts, mongo/MongoReplicaFixture.ts, ...) were instrumented and reported to Codecov as production code, understating the patch-coverage denominator correction the gate already applies elsewhere. Add '!src/**/__tests/**' so this package's own coverage collection matches the gate's definition of test code. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(sdk): pin HTTPS lookup facilitator transport bounds Cover the cancellation and resource-limit branches of HTTPSOverlayLookupFacilitator: a pre-aborted caller signal short-circuits before any fetch, an abort racing a non-ok response reports cancellation and drains the body, JSON and octet-stream output counts are rejected above the caller's budget (including a negative varint count), atomic BEEF extraction stops once the extracted bytes outrun the response budget, and an in-flight decode observes a mid-stream cancellation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(sdk): pin lookup query resource, cancellation and host-policy bounds Exercise the uncovered fail-closed branches of LookupResolver.query$: aggregation stopping at maxOutputs across hosts, the evidenceLimits shorthand defaulting its byte budget, the bounded candidate scan (scan window, malformed entries, maxHosts), per-source quotas for additional hosts, SLAP discovery refusing bytes that would breach maxTotalBytes, byte reports arriving after cancellation, cancellation dropping an in-flight peer answer and skipping queued hosts, idempotent double cancellation, the concurrent-query ceiling, over-budget answers being dropped without blaming the host, the maxTrackers budget, advertisement-map eviction, and a broader host cache surviving a tighter rediscovery. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): pin MongoPayloadStore fail-closed and fencing behavior Adds real jest tests for previously-uncovered MongoPayloadStore branches: pin re-add no-op, claimGarbage/finishGarbage edge cases (never-published digest, afterDeleteClaim hook, ownership-metadata drift), lease-stolen fencing before the ready CAS and before GridFS staging, wrong explicit txid on both the fresh-upload and already-ready paths, GridFS crash-boundary hooks, abandonUpload's guard against deleting a concurrently-published file, digest-mismatch-after-spillover cleanup, stale inline-upload recovery, an expired crashed reservation reclaiming and retiring its orphaned GridFS file, and the abort-signal fallback messages used when a signal's `reason` is unset. Also stresses a many-way concurrent first-time publish of the same digest and a corrupted-in-place staged GridFS chunk to exercise the duplicate-key race and post-upload verification failure paths. Uncovered lines for this file drop from 57 to 21 (lcov, line+branch). Remaining lines are either dead code given existing invariants (the per-chunk maxPayloadBytes check, verifyGridFs's non-byte-chunk guard, the "stream writer missing" guard) or depend on timing-sensitive races with no exposed test seam (recoverStaleUpload's current-state guard, the exact instant a GridFS write signals backpressorem). A test also surfaced a real defect, reported separately: finishGarbage's FileNotFound-swallow regex never matches the mongodb driver's actual "File not found for id X" message, so a benign already-deleted-file race during cleanup always rethrows instead of being swallowed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): pin MongoTransactionRunner claim races and reconcile edges Adds real jest tests for previously-uncovered MongoTransactionRunner branches: many concurrent fresh claims on the same operation id (forces the duplicate-key insert race and the "lost the claim" observer path), a claim-row insert failure whose error code is not a duplicate key (must propagate, not be swallowed), reconcile() with no attemptId against a never-created operation (the "unlocated" fallback), reconcile() with a mismatched attemptId against a genuinely pending claim, and commit failures whose deadline elapses mid-retry-loop (the budget-aborted break, verified via a real blocked commitTransaction). Uncovered lines for this file drop from 18 to 9 (lcov, line+branch). Remaining lines depend on races or invariants not reachable from the public API: Budget's manual deadline check is shadowed by the earlier throwIfAborted() once the matching timer fires; abort()'s "already out of transaction" branch has no path that reaches it before the catch that calls it; the post-insert re-read uses a majority/primary read right after an acknowledged write, so it can't observe a miss without breaking the replica set's own consistency guarantee; and the ownership-fencing mismatches and the "operation row disappeared" branches require deleting a submission-operation row, which nothing in this class ever does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): pin MongoSchema bootstrap idempotency and failure paths Adds real jest tests for previously-uncovered MongoSchema branches: concurrent bootstraps of a fresh database (races the NamespaceExists and duplicate-ledger swallow paths in createCollectionIfMissing, ensureGridFs, and ensureLedger), a concurrent re-bootstrap after dropping just the GridFS bucket collections, an existing collection whose validator matches but omits an explicit collation (the collation-compatible-by-omission branch), an index-creation failure whose error code is not one of the benign conflict codes, and a ledger insert failure whose error code is not a duplicate key (both must propagate rather than being swallowed). Uncovered lines for this file drop from 11 to 10 deterministically; the concurrent-bootstrap races additionally land on the NamespaceExists swallow branches in some runs but not every run, since they depend on genuine scheduling of concurrent createCollection calls. The remaining lines are dead code given the surrounding invariants (decodeMongoUint64's re-encode check, the unused `string()` default length and `allowAdditional` option on module-private helpers no call site ever exercises) or require racing a second writer against transactionalProbe's own read-then-CAS window, which was not attempted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(overlay-mongo): tolerate an already-deleted GridFS file in finishGarbage finishGarbage is documented as safe to retry, but it only swallowed errors matching /FileNotFound/. The MongoDB driver reports a missing file as MongoRuntimeError "File not found for id <id>", which never matched, so a finisher that lost the race to a concurrent retry rethrew and left the payload row in 'deleting'. Match the driver's actual message. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
sirdeggen
added a commit
that referenced
this pull request
Sep 19, 2026
* fix(wallet-toolbox): verify overlay identity evidence * feat(overlay): define optional admission and recovery contracts * fix(sdk): preserve safe integer Merkle path offsets * chore(sdk): align candidate version registry * feat(sdk): add dynamic overlay lookup discovery and bounded host scheduling Subscribe to SLAP tracker updates during an active query so later advertised hosts can join a fair, concurrency-bounded queue. Read lookup bodies incrementally with honest resource-limit accounting, keep trackers as routing sources, and leave raw query$ unverified while exposing the C02 onEvidence intake seam. Preserves 2s/5s delays, reputation/backoff, query/query$/freeform APIs, and existing CORS/public lookup request headers. * feat(overlay): add opt-in Mongo payload and schema foundation Add replica-set schema bootstrap, content-addressed payload publication, transactionally guarded references/GC, and majority-commit outcome helpers behind an optional mongodb peer. This is not an AdmissionStorage adapter and does not change Engine or Knex defaults. * fix(sdk): close C02/C03 evidence coordinator review findings Preserve per-entry script results so a rejected sibling cannot poison a shared ancestor. LocalChainTracker tokens use participating sources only and fail closed on missing identity. Reset ownership is rechecked after dispose before destructive hooks. Built-in remote ChainTracks clients advertise reorg-event capability explicitly. Services.getChainTracker publishes one coalesced wrapper before yielding. * feat(overlay): harden BASM protocol validation and Go interop Validate untrusted BASM peer JSON, bind remote anchors to local canonical headers, and independently check admitted-list block indices before historical admission. Keep automatic BASM sync disabled and leave durable recovery jobs for B02. Include independent BRC-136 vector checks and localhost TS client tests against the Go five-method read/serving surface. * fix(wallet-toolbox): re-read BHS canonical headers for root checks BHServiceClient no longer caches the queried merkle root. Validity is decided from a freshly read header, so a false result cannot invert on retry and a reorg cannot reuse a stale positive. * fix(sdk): keep overlay host cache and query() deadline honest Store discovery bounds on the SLAP host cache so a tighter-limit query cannot freeze a truncated host set for a later broader attempt. Throw the historical no-competent-hosts error from query()/queryDetailed when a deadline expires before any host is admitted; query$ still emits the deadline snapshot. * fix(overlay): compare Mongo uint32 indexes as integers Collection validators used lexicographic $lte on unpadded outputIndex strings, which rejected legal indexes such as "9". Bound uint32 fields with $toLong on outputs, consumption edges, GASP nodes, and SHIP/SLAP. * fix(overlay): bind BASM admitted indices without coinbase maturity Verify claimed admitted-list block indices against the compound Merkle path even when every remote txid is already local. Check inclusion with the chain-tracker root at the claimed height instead of MerklePath.verify, which also enforces coinbase 100-block spendability. * fix(sdk): encode overlay discovery cache keys with stringifyBRC100 LookupResolver is a BRC-100 JSON boundary file, so the in-flight discovery key cannot use JSON.stringify. stringifyBRC100 preserves the same service-plus-limit tuple without changing cache cover semantics. * fix(sonar): evidence coordinator and chaintracks findings Drop redundant optional-undefined on lookup evidence limits, extract parseEvidence and Chaintracks header-retry helpers below S3776, and rename coordinator catch params to error_. * fix(sonar): overlay BASM protocol findings Extract BASM reconcile helpers so cognitive complexity stays at or below 15, prefer optional chaining on nullable anchors, and invert the advertised response-length comparison. * fix(sonar): overlay lookup discovery complexity and style Extract query/session and host-tracking helpers so cognitive complexity stays within Sonar's limit, and clear the remaining overlay-tools findings without changing lookup behavior. * fix(sonar): overlay mongo and admission findings Brand StorageUint64, extract high-complexity helpers, and apply Sonar-preferred optional chaining, regex, clone, and sort APIs without changing payload identity or uint32 $toLong validators. * fix(sonar): throw lookup abort instead of Promise.reject S7746 on LookupResolver: prefer throw over return Promise.reject in the facilitator start chain. * fix(wallet-toolbox): skip overlay identity when services are absent discoverOverlayCertificates called getServices(), which throws on wallets constructed without services and broke BRC-100 discoverBy* conformance. Missing chain context now yields no identities instead of throwing. * fix(ci): override js-yaml 3.15.2 for GHSA-2883 Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high. * fix(ci): override js-yaml 3.15.2 for GHSA-2883 Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high. * fix(ci): align sdk 2.5.0 health baseline and stack facts Merge from main left baselines.json on 2.4.2 while package.json is 2.5.0. * fix(ci): override js-yaml 3.15.2 for GHSA-2883 Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high. * fix(ci): override js-yaml 3.15.2 for GHSA-2883 Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high. * fix(ci): record js-yaml override removal rehearsal Health requires overrideRemovalReview.retainedCount to match the live registry. * fix(ci): record js-yaml override removal rehearsal Health requires overrideRemovalReview.retainedCount to match the live registry. * fix(ci): record js-yaml override removal rehearsal Health requires overrideRemovalReview.retainedCount to match the live registry. * fix(ci): record js-yaml override removal rehearsal Health requires overrideRemovalReview.retainedCount to match the live registry. * fix(ci): ratchet override count for js-yaml 3.15.2 * fix(ci): ratchet override count for js-yaml 3.15.2 * fix(ci): ratchet override count for js-yaml 3.15.2 * fix(ci): ratchet override count for js-yaml 3.15.2 * fix(ci): raise SDK and wallet platform bundle budgets LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates. * fix(ci): raise SDK and wallet platform bundle budgets LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates. * fix(ci): raise SDK and wallet platform bundle budgets LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates. * fix(ci): raise SDK and wallet platform bundle budgets LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads over the previous raw-size gates. * fix(ci): raise remaining SDK-consumer bundle budgets Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth. * fix(ci): raise remaining SDK-consumer bundle budgets Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth. * fix(ci): raise remaining SDK-consumer bundle budgets Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth. * fix(ci): raise remaining SDK-consumer bundle budgets Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the evidence/discovery helper payload growth. * fix(ci): raise wallet client esbuild brotli budget * fix(ci): raise wallet client esbuild brotli budget * fix(ci): raise wallet client esbuild brotli budget * fix(ci): raise wallet client esbuild brotli budget * fix(ci): raise did-client UMD budget for SDK helper growth * fix(ci): raise did-client UMD budget for SDK helper growth * fix(ci): raise did-client UMD budget for SDK helper growth * fix(ci): raise did-client UMD budget for SDK helper growth * fix(ci): prettier browser budget JSON * fix(ci): prettier browser budget JSON * fix(ci): prettier browser budget JSON * fix(ci): prettier browser budget JSON * fix(ci): raise did-client esbuild budget * fix(ci): raise did-client esbuild budget * fix(ci): raise did-client esbuild budget * fix(ci): raise did-client esbuild budget * fix(ci): raise did-client vite budget * fix(ci): raise did-client vite budget * fix(ci): raise did-client vite budget * fix(ci): raise did-client vite budget * fix(ci): raise SDK vite and esbuild compressed budgets * fix(ci): raise SDK vite and esbuild compressed budgets * fix(ci): raise SDK vite and esbuild compressed budgets * fix(ci): raise SDK vite and esbuild compressed budgets * fix(wallet-toolbox): throw on overlay identity forceRefresh without services Contact discovery still works without services. forceRefresh bypasses contacts and still requires a chain tracker, matching the existing test. * test(overlay): cover BASM remote limits for patch coverage * fix(sonar): extract overlay chain-tracker guard from Wallet discovery S3776: discoverOverlayCertificates was 17 after the forceRefresh services check. The missing-services path is now requireOverlayChainTracker. * test(sdk): cover overlay discovery helpers for patch coverage * test(overlay): cover mongo storage helpers for patch coverage * fix(sonar): use toHaveLength in LookupResolver discovery tests * test(sdk): cover chain tracker and evidence helpers for patch coverage * fix(ci): parent and finalize Codecov coverage reports * fix(ci): parent and finalize Codecov coverage reports * fix(ci): parent and finalize Codecov coverage reports * docs: reverify resource and TTN rollout gates * docs: reverify resource and TTN rollout gates * docs: reverify resource and TTN rollout gates * fix(ci): require HTTPS for Codecov polling * fix(ci): require HTTPS for Codecov polling * fix(ci): require HTTPS for Codecov polling * fix(ci): raise SDK esbuild raw browser budget * test(sdk): cover invalid evidence intake limits * test(overlay): cover uint64 payload length boundary * test(overlay): advertise the Go interop listener as explicit loopback * fix(ci): remove duplicate workspace override * fix: make overlay admission runtime portable * fix: avoid eager fetch dependency in broadcaster * fix: avoid eager fetch dependency in broadcaster * fix: defer unavailable fetch failure until send * fix: defer unavailable fetch failure until send * fix: defer unavailable fetch failure until send * test: provide fetch spy target in node * test: match complete admission acknowledgment * chore(overlays): re-derive release metadata after the ordered merges Refreshes the overlay package documentation dates for the four packages this change bumps and regenerates the derived reference documents so the stack facts and package API migration tables match the merged versions. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(overlays): re-derive release metadata after the ordered merges Apply this PR's own SemVer impact on top of what the predecessor branch leaves, and restore the cumulative release prose the branch had regressed. - Versions: @bsv/overlay 2.5.0, @bsv/overlay-express 2.7.0, @bsv/overlay-discovery-services 2.2.3, @bsv/overlay-topics 1.8.2, with matching repository-health baselines and docs frontmatter. - governance/package-release-notes.json: keep main's and the predecessor's published versions and cumulative text, and append only this PR's BASM sentences for the four overlay packages. - docs/packages/sdk/bsv-sdk.md: make the Merkle-path candidate note version-neutral so it no longer names a superseded release. - Regenerate docs/reference/package-api-migrations.md, stack-facts.md and service-operations.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): keep the evidence branch esbuild budget at its measured ceiling The main integration merge raised the SDK esbuild raw ceiling to 600000, a value that belongs to the lookup discovery change (#518). This branch measured 590000; restore it so the diff carries only its own budget. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): scope lookup discovery docs and budget to this change Restore the eleven ts2md reference pages for modules this change does not touch to the incoming copies; the generator run had imported unrelated drift that already exists on main. Keep the regenerated overlay-tools page. Reinstate this change's own 600000 esbuild raw ceiling after restacking on the corrected evidence branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(sdk): record the Merkle path offset correction in the release notes The ordered-merge integration took the incoming @bsv/sdk release-note entry verbatim and dropped this change's sentences. Append them to the cumulative candidate text and regenerate the package migration table. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): reject cancelled lookups instead of answering empty query() and queryDetailed() flatten query$ to its first snapshot. The terminal gate only threw when hostCount was 0, and explicitly skipped the throw for terminalReason 'cancelled', so a caller abort resolved as { type: 'output-list', outputs: [] }. At any host count an aborted attempt was indistinguishable from an authoritative "this service has no outputs" answer, which is exactly the completion-vs-empty confusion queryDetailed exists to prevent. Throw lookupAbortError() before the host-count gate whenever the terminal snapshot reports 'cancelled'. query$ is unchanged and still emits its terminalReason: 'cancelled' snapshot for progressive callers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): keep discovery resource limits out of the no-hosts error chargeLookupDiscoveryBytes can exhaust maxTotalBytes while SLAP trackers are still being read, before any host is admitted. That leaves the terminal snapshot with hostCount 0, terminalReason 'resource-limit' and the limit recorded in limitsHit. queryDetailed()'s hostCount gate did not look at terminalReason, so it threw the historical "No competent <network> hosts found by the SLAP trackers" Error and discarded both fields. Security-sensitive Promise callers could not tell "the trackers named no hosts" from "this attempt ran out of its own byte budget". Branch the gate on terminalReason: 'resource-limit' throws LookupResourceLimitError carrying the first limit that was hit, while 'deadline' and a settled empty discovery keep the historical error and message unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(overlay): keep SPV verification on historical-tx submissions A merge on this branch widened the submit() SPV guard from `mode !== 'historical-tx-no-spv'` to `mode === 'current-tx'`. That removed `tx.verify(this.chainTracker)` from every `historical-tx` submission, not just from the BASM admission path that needed it. `historical-tx` is a public submission mode, so any caller could get an unverified transaction applied to a topic. Root cause: the guard was widened to work around a symptom. BASM admission proves inclusion independently (`chainTracker.isValidRootForHeight` plus the canonical-header binding) and must not re-apply `MerklePath.verify`'s coinbase 100-block spendability rule, but the correct lever for that is the existing `historical-tx-no-spv` mode on the BASM call site, not a relaxed global guard. Restores the original guard and switches the BASM admission in `fetchBASMMissingTransactions` to `historical-tx-no-spv`. The other effects of that mode are correct here: propagation is skipped (already the case for `historical-tx`), and although the broadcast block is entered, every BASM-admitted transaction carries its extracted Merkle path, so `broadcastAcceptedSubmission` synthesises a local success and never reaches the network. Topic managers now observe `historical-tx-no-spv` for BASM admissions; no bundled manager branches on the mode. Tests: the coinbase fixture no longer mocks `Engine.submit`, so the young offset-0 proof is admitted through the production path, and a new test proves a GASP-style `historical-tx` submission with an invalid proof is still rejected. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): refuse redirects on overlay lookup and SLAP discovery requests performLookupRequest built its RequestInit without a redirect field, so fetch defaulted to 'follow'. normalizeLookupHost and the https: guard in lookup() validate the advertised URL only; neither runs again on the final URL. A SLAP-advertised HTTPS host could therefore answer the lookup POST with a 307/308 and have the serialized stringifyBRC100({ service, query }) body re-sent to http:, loopback or a link-local address. This PR queries every eligible advertised host, so the hop sat on the hot path for untrusted SLAP domains, and the SLAP tracker discovery requests reach the network through this same method. Set redirect: 'error' so the transport fails closed. fetch rejects, and the existing failure path records the advertised host as an ordinary availability failure instead of crashing the query or following the hop. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): gate Go interop solely on BASM_GO_OVERLAY_SERVICES `resolveGoWorktree()` tried the `BASM_GO_OVERLAY_SERVICES` environment variable and then fell back to the literal path `/Users/personal/git/go/worktrees/go-overlay-services-basm`. That fallback made the suite run on exactly one workstation and `describe.skip` everywhere else, including CI, so the B01 protocol was never actually spoken by any job that claimed to cover it, and the skip was silent. Root cause: the checkout location was treated as a discovery problem with a convenient default instead of an explicit input. Resolution is now the environment variable alone. When it is unset the suite is skipped under a title that names the variable; when it is set but does not resolve to a go-overlay-services checkout the resolver throws, so a job configured for Go interop fails closed rather than quietly skipping. Tests: a new BASMGoInteropGating suite asserts the gate from the interop source, which is the only machine-independent way to catch this class of regression - a behavioural assertion agrees with a workstation fallback on every machine that lacks that path, which is precisely where the fallback hides. The fixtures README now states that Go interop does not run by default and records the fixture provenance as a repository-relative path. Verified locally: with BASM_GO_OVERLAY_SERVICES=/Users/personal/git/go/worktrees/go-overlay-services-basm the interop suite builds the Go host and passes (1/1); with a non-checkout path the suite fails to run with the explanatory error; unset, it skips. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(overlay-mongo): serialize pin re-add expiry with GC reclaim addReference's idempotent re-add branch only re-validated the payload was still ready; it never touched the existing (ownerKind, ownerId, slot) pin row's expiresAt. Extending a pin therefore silently kept the stale expiry, and claimGarbage counted an already-expired pin as not-live via $expr {$gt:['$expiresAt','$$NOW']} but never deleted that expired pin document. The unique slot row survived expiry and kept short-circuiting future addReference calls on that slot, so a caller could be told content stayed pinned while GC moved the payload ready -> deleting underneath it. Add refreshPinExpiry: called after the existing {_id: payloadId, state: 'ready'} CAS matches (so it stays serialized against a concurrent GC claim in the same transaction), it reactivates an expired pin with the caller's new expiry, allows extending a still- live pin to a later expiry, and rejects an attempt to shorten a live pin's expiry rather than silently ignoring it. claimGarbage now deletes every expired pin reference row for the payload in the same transaction as its ready -> deleting claim, so a stale slot cannot outlive the payload it named. Document the re-add/extend/reactivate/shorten rule in specs/overlay/mongo-v1.md next to the existing pin semantics. Regression tests (mongodb-memory-server replica fixture) added to src/__tests/mongo/MongoPayloadStore.test.ts: - re-adding a pin slot extends its expiry but never shortens it silently - re-adding an expired pin slot reactivates it and makes the payload live again - claimGarbage deletes the expired pin reference row inside the same transaction as its claim All three were confirmed to fail against the pre-fix source for the reason described above (verified by temporarily reverting this file and re-running the suite) before the fix was applied. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(overlay): record the BASM admission submission mode Follow-up to 7584d4e, which moved BASM admission from the public `historical-tx` submission mode to `historical-tx-no-spv` so that restoring the SPV guard does not re-apply MerklePath.verify's coinbase 100-block spendability rule to an independently proven admission. The submission mode is passed through to `TopicManager.identifyAdmissibleOutputs`, so it is observable by every topic manager that branches on it. The release prose described the inclusion rule but not the mode a manager now sees, which left implementors without the one externally visible consequence of the change. Adds that sentence to the @bsv/overlay release note and the package CHANGELOG entry, and regenerates docs/reference/package-api-migrations.md from the governance source. No code or behaviour change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay-mongo): cover the too-small declared length reclaim path The reviewer asked for a regression test on the reclaim path taken when publish fails because the caller declared a too-small byteLength: consumePayloadChunk rejects the stream once it exceeds the declared length, abandonUpload marks the payload row 'deleted', and a retry with the correct digest and length must reclaim that row through tryClaimUpload (which already recomputes byteLength from the new input when the prior state is 'deleted') and finish 'ready' with the corrected byteLength stored. This path was already implemented and covered indirectly (a sibling test manually sets state: 'deleted' with a wrong byteLength via direct DB mutation), but there was no test driving the actual publish/fail/ retry sequence end to end. This is coverage-only: the reclaim logic in tryClaimUpload predates this change and needed no fix, so there is no meaningful pre-fix state to fail against in this worktree; the new test passes against the current code, which is the expected outcome for confirming existing, correct behavior. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(sdk): re-queue displaced same-txid candidates instead of dropping them TransactionEvidenceCoordinator.run() shifts the next candidate off a job's queue before calling attempt(). If every concurrency slot is already in use (including slots still reserved by non-abortable, already-finished attempts; see the "does not free a non-abortable backend slot..." test), attempt() threw a plain TransactionEvidenceError('limit') and called the caller's `settled` callback. run()'s catch treated that exactly like a genuine verification failure: the candidate was gone (already shift()ed) and never pushed back, so a same-txid alternate candidate that had already been admitted to the job could be silently discarded without ever being tried, even though it was perfectly valid. Fix: attempt() now throws a dedicated ConcurrencyLimitSignal when the operation never started, and does not invoke `settled` (nothing was consumed, so no byte/candidate bookkeeping should be released). run()'s candidate loop catches that signal specifically, unshifts the candidate back onto job.candidates, marks the job not-running, and returns without looping or retrying itself. Re-admission is driven solely by pump() being invoked again when some other active attempt settles, so this cannot spin or busy-wait for a slot. Added a regression test that builds a genuine concurrency race: a non-abortable "ghost" attempt (cancelled while its script verification is still in flight) holds one of two slots, a job with two same-txid candidates (one invalid, one valid) takes the other, and a third waiting job is admitted into the slot freed by the first candidate's failure before the job's own retry can reclaim it. Before the fix this made the valid alternate candidate reject with 'limit'; the test fails for that reason on the old code and passes with the fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(wallet-toolbox): implement BHServiceClient.findChainTipHash Every TransactionEvidenceCoordinator attempt calls ChaintracksChainTracker.getVerificationContextToken(), which requires chaintracks.findChainTipHash(). BHServiceClient implemented that method as `throw new Error('Not implemented')` even though it already exposes a working findChainTipHeader() against /api/v1/chain/tip/longest. Any wallet using a BHServiceClient as options.chaintracks, or as a LocalChainTracker participating source, broke on every verification attempt. Fix: implement findChainTipHash() by delegating to the existing findChainTipHeader() and returning its hash, matching the same pattern GoChaintracksServiceClient already uses for the same interface method. No other ChaintracksClientApi contract surface changes. Added a regression test on BHServiceClient confirming findChainTipHash() resolves to findChainTipHeader()'s hash instead of throwing, and a regression test on ChaintracksChainTracker confirming getVerificationContextToken() succeeds end-to-end when backed by a BHServiceClient. Both fail with "Not implemented" on the old code and pass with the fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: changelog entries for F1/F2 review-finding fixes Document the TransactionEvidenceCoordinator candidate-requeue fix and the BHServiceClient.findChainTipHash implementation in their packages' CHANGELOG.md, matching this repo's convention of recording behavior- affecting fixes there alongside the commit history. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(sdk): state why a cached evidence recheck cannot be displaced The concurrency-limit signal is handled only in the candidate loop. The cached-positive recheck does not need the same handling: run() is reached solely from pump(), which checks the attempt limit synchronously before starting the job, and no await separates that check from the recheck attempt. Record the invariant next to the call so a later refactor of pump() or run() revisits it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(wallet): set platform budgets from measured evidence-branch bundles The ceilings carried on this branch were authored before main grew (#534, #547), so the merged wallet bundles exceeded them and CI stopped at the first over-budget dimension. Measured from the exact packed dependency graph at 83c0670 (JS bundle sizes verified byte-identical between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio recorded in #547): client Vite 1,759,717 raw / 416,284 gzip / 324,287 brotli client esbuild 1,372,320 raw / 378,741 gzip / 302,937 brotli mobile Metro 1,811,581 raw / 461,355 gzip / 355,871 brotli mobile Hermes 3,688,871 raw / 1,502,972 gzip / 1,166,821 brotli Only the dimensions that exceeded are raised, with about 0.25% headroom (1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(budgets): set ceilings from measured lookup discovery bundles Bounded discovery, the host queue and the streaming response reader grow every bundle that embeds @bsv/sdk. Measured from the exact packed dependency graph at the top of the ordered overlay wave (max of 5eef0fc and 9077394; JS bundle sizes verified byte-identical between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio from #547): @bsv/sdk UMD 591,705 raw / 168,531 gzip / 140,485 brotli @bsv/simple Vite 752,122 raw / 182,787 gzip / 150,699 brotli wallet client Vite 1,782,018 raw / 421,917 gzip / 328,724 brotli wallet client esbuild 1,389,928 raw / 383,620 gzip / 306,783 brotli wallet mobile Metro 1,834,234 raw / 466,804 gzip / 360,591 brotli wallet mobile Hermes 3,725,120 raw / 1,520,644 gzip / 1,181,384 brotli Only the dimensions that exceeded are raised, with about 0.25% headroom (1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance). Later branches in the wave move these bundles by under 150 bytes, so they inherit these ceilings unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): name the historical-tx SPV test for what it covers GASP finalization already submits with historical-tx-no-spv; its historical-tx calls are dry-run admissibility checks. The restored guard protects public Engine.submit callers, so say that in the test title. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): exclude src/__tests support modules from coverage The repository's patch-coverage gate (scripts/patch-coverage.mjs, TEST_PATH pattern /(?:^|\/)__tests(?:__)?(?:\/|$)/) treats both `__tests/` and `__tests__/` directories as test code. This package's jest collectCoverageFrom only excluded `__tests__/**`, so the non-`.test.ts` support modules this PR adds under src/__tests/ (admission/ReferenceAdmissionStorage.ts, admission/AdmissionStorageContract.ts, mongo/MongoCommitResponseProxy.ts, mongo/MongoReplicaFixture.ts, ...) were instrumented and reported to Codecov as production code, understating the patch-coverage denominator correction the gate already applies elsewhere. Add '!src/**/__tests/**' so this package's own coverage collection matches the gate's definition of test code. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): share the BASM reconciliation peer fixture Move the fake-peer fixture out of BASMReconciliation.test.ts so further fail-closed suites can drive the same engine, storage, chain tracker and fetch mock without duplicating them. No test behaviour changes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): pin BASM_UNSUPPORTED provider guards Each BASM read endpoint fails closed with a TypeError carrying code 'BASM_UNSUPPORTED' when the storage backend does not implement the optional method behind it, and the capability check runs before the argument-shape guard on compound Merkle paths. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): cover BASM tip-comparison and error reporting paths Exercises reconcileBASMWithPeer's not-ahead branch end to end: two empty topics match without a range request, an equal matching tip is only accepted after its local anchor is re-verified against the canonical header, a higher local tip reports divergence, and a tip whose anchor is missing, unsupported, TAC-mismatched or non-canonical fails closed. Also pins the report surface: "matched" once the page catches the local tip up, a stringified non-Error failure with no invented errorCode, and BASM_UNSUPPORTED surfacing through startBASMSync. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): cover BASM compound-proof position binding Adds the canonical-position cases the reconciler fails closed on: a phantom duplicate at the right edge of an odd-width block is accepted but the same node is rejected once the canonical width is even, a BUMP that omits an admitted leaf is rejected before any raw transaction is fetched, a non-canonical root is rejected by the chain tracker, canonical-count assurance is recorded even when nothing needs fetching, and an anchor that vanishes from the peer blocks admission. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): cover BASM forward-page prefix and assurance reporting Drives a multi-block fake peer to pin the anchor-page contract: a page truncated short of its requested target and a page that skips the height after the local tip are both rejected, a peer many pages ahead is followed one capped page at a time without re-downloading anchors that already match locally, and a page mixing canonical-count and encoded-offset blocks reports the weaker assurance for the whole attempt. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(sdk): pin HTTPS lookup facilitator transport bounds Cover the cancellation and resource-limit branches of HTTPSOverlayLookupFacilitator: a pre-aborted caller signal short-circuits before any fetch, an abort racing a non-ok response reports cancellation and drains the body, JSON and octet-stream output counts are rejected above the caller's budget (including a negative varint count), atomic BEEF extraction stops once the extracted bytes outrun the response budget, and an in-flight decode observes a mid-stream cancellation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * style(overlay): prettier-format the new BASM test suites Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(sdk): pin lookup query resource, cancellation and host-policy bounds Exercise the uncovered fail-closed branches of LookupResolver.query$: aggregation stopping at maxOutputs across hosts, the evidenceLimits shorthand defaulting its byte budget, the bounded candidate scan (scan window, malformed entries, maxHosts), per-source quotas for additional hosts, SLAP discovery refusing bytes that would breach maxTotalBytes, byte reports arriving after cancellation, cancellation dropping an in-flight peer answer and skipping queued hosts, idempotent double cancellation, the concurrent-query ceiling, over-budget answers being dropped without blaming the host, the maxTrackers budget, advertisement-map eviction, and a broader host cache surviving a tighter rediscovery. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): pin MongoPayloadStore fail-closed and fencing behavior Adds real jest tests for previously-uncovered MongoPayloadStore branches: pin re-add no-op, claimGarbage/finishGarbage edge cases (never-published digest, afterDeleteClaim hook, ownership-metadata drift), lease-stolen fencing before the ready CAS and before GridFS staging, wrong explicit txid on both the fresh-upload and already-ready paths, GridFS crash-boundary hooks, abandonUpload's guard against deleting a concurrently-published file, digest-mismatch-after-spillover cleanup, stale inline-upload recovery, an expired crashed reservation reclaiming and retiring its orphaned GridFS file, and the abort-signal fallback messages used when a signal's `reason` is unset. Also stresses a many-way concurrent first-time publish of the same digest and a corrupted-in-place staged GridFS chunk to exercise the duplicate-key race and post-upload verification failure paths. Uncovered lines for this file drop from 57 to 21 (lcov, line+branch). Remaining lines are either dead code given existing invariants (the per-chunk maxPayloadBytes check, verifyGridFs's non-byte-chunk guard, the "stream writer missing" guard) or depend on timing-sensitive races with no exposed test seam (recoverStaleUpload's current-state guard, the exact instant a GridFS write signals backpressorem). A test also surfaced a real defect, reported separately: finishGarbage's FileNotFound-swallow regex never matches the mongodb driver's actual "File not found for id X" message, so a benign already-deleted-file race during cleanup always rethrows instead of being swallowed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): pin MongoTransactionRunner claim races and reconcile edges Adds real jest tests for previously-uncovered MongoTransactionRunner branches: many concurrent fresh claims on the same operation id (forces the duplicate-key insert race and the "lost the claim" observer path), a claim-row insert failure whose error code is not a duplicate key (must propagate, not be swallowed), reconcile() with no attemptId against a never-created operation (the "unlocated" fallback), reconcile() with a mismatched attemptId against a genuinely pending claim, and commit failures whose deadline elapses mid-retry-loop (the budget-aborted break, verified via a real blocked commitTransaction). Uncovered lines for this file drop from 18 to 9 (lcov, line+branch). Remaining lines depend on races or invariants not reachable from the public API: Budget's manual deadline check is shadowed by the earlier throwIfAborted() once the matching timer fires; abort()'s "already out of transaction" branch has no path that reaches it before the catch that calls it; the post-insert re-read uses a majority/primary read right after an acknowledged write, so it can't observe a miss without breaking the replica set's own consistency guarantee; and the ownership-fencing mismatches and the "operation row disappeared" branches require deleting a submission-operation row, which nothing in this class ever does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(overlay): pin MongoSchema bootstrap idempotency and failure paths Adds real jest tests for previously-uncovered MongoSchema branches: concurrent bootstraps of a fresh database (races the NamespaceExists and duplicate-ledger swallow paths in createCollectionIfMissing, ensureGridFs, and ensureLedger), a concurrent re-bootstrap after dropping just the GridFS bucket collections, an existing collection whose validator matches but omits an explicit collation (the collation-compatible-by-omission branch), an index-creation failure whose error code is not one of the benign conflict codes, and a ledger insert failure whose error code is not a duplicate key (both must propagate rather than being swallowed). Uncovered lines for this file drop from 11 to 10 deterministically; the concurrent-bootstrap races additionally land on the NamespaceExists swallow branches in some runs but not every run, since they depend on genuine scheduling of concurrent createCollection calls. The remaining lines are dead code given the surrounding invariants (decodeMongoUint64's re-encode check, the unused `string()` default length and `allowAdditional` option on module-private helpers no call site ever exercises) or require racing a second writer against transactionalProbe's own read-then-CAS window, which was not attempted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(overlay-mongo): tolerate an already-deleted GridFS file in finishGarbage finishGarbage is documented as safe to retry, but it only swallowed errors matching /FileNotFound/. The MongoDB driver reports a missing file as MongoRuntimeError "File not found for id <id>", which never matched, so a finisher that lost the race to a concurrent retry rethrew and left the payload row in 'deleting'. Match the driver's actual message. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(overlay): reconcile BASM heights where the topic admitted nothing BRC-136 defines an empty topic at a height as k = 0 with R = 32 zero bytes, and rebuildTopicAnchorChain anchors every height that way. The reconciler accepted such an anchor and then requested a compound Merkle path for an empty txid list, which BASMRemote rejects ("At least one BASM proof txid is required"). Any node behind a peer aborted at the first height the topic admitted nothing, which is most heights. After the root and count check, an empty admitted list has nothing to bind or fetch: count the height as checked and skip the proof and raw transaction round trip. It diverges only when this node admitted transactions at that height. A zero-count anchor with a non-zero root is still rejected. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
sirdeggen
added a commit
that referenced
this pull request
Sep 19, 2026
* fix(wallet-toolbox): verify overlay identity evidence
* feat(overlay): define optional admission and recovery contracts
* fix(sdk): preserve safe integer Merkle path offsets
* chore(sdk): align candidate version registry
* feat(sdk): add dynamic overlay lookup discovery and bounded host scheduling
Subscribe to SLAP tracker updates during an active query so later
advertised hosts can join a fair, concurrency-bounded queue. Read lookup
bodies incrementally with honest resource-limit accounting, keep
trackers as routing sources, and leave raw query$ unverified while
exposing the C02 onEvidence intake seam.
Preserves 2s/5s delays, reputation/backoff, query/query$/freeform APIs,
and existing CORS/public lookup request headers.
* feat(overlay): add opt-in Mongo payload and schema foundation
Add replica-set schema bootstrap, content-addressed payload publication,
transactionally guarded references/GC, and majority-commit outcome helpers
behind an optional mongodb peer. This is not an AdmissionStorage adapter
and does not change Engine or Knex defaults.
* fix(sdk): close C02/C03 evidence coordinator review findings
Preserve per-entry script results so a rejected sibling cannot poison a
shared ancestor. LocalChainTracker tokens use participating sources only
and fail closed on missing identity. Reset ownership is rechecked after
dispose before destructive hooks. Built-in remote ChainTracks clients
advertise reorg-event capability explicitly. Services.getChainTracker
publishes one coalesced wrapper before yielding.
* feat(overlay): harden BASM protocol validation and Go interop
Validate untrusted BASM peer JSON, bind remote anchors to local canonical
headers, and independently check admitted-list block indices before
historical admission. Keep automatic BASM sync disabled and leave durable
recovery jobs for B02.
Include independent BRC-136 vector checks and localhost TS client tests
against the Go five-method read/serving surface.
* fix(wallet-toolbox): re-read BHS canonical headers for root checks
BHServiceClient no longer caches the queried merkle root. Validity is
decided from a freshly read header, so a false result cannot invert on
retry and a reorg cannot reuse a stale positive.
* fix(sdk): keep overlay host cache and query() deadline honest
Store discovery bounds on the SLAP host cache so a tighter-limit query
cannot freeze a truncated host set for a later broader attempt. Throw
the historical no-competent-hosts error from query()/queryDetailed when
a deadline expires before any host is admitted; query$ still emits the
deadline snapshot.
* fix(overlay): compare Mongo uint32 indexes as integers
Collection validators used lexicographic $lte on unpadded outputIndex
strings, which rejected legal indexes such as "9". Bound uint32 fields
with $toLong on outputs, consumption edges, GASP nodes, and SHIP/SLAP.
* fix(overlay): bind BASM admitted indices without coinbase maturity
Verify claimed admitted-list block indices against the compound Merkle
path even when every remote txid is already local. Check inclusion with
the chain-tracker root at the claimed height instead of MerklePath.verify,
which also enforces coinbase 100-block spendability.
* feat(overlay): add opt-in Mongo admission commit and Engine submit path
Wire Engine.submit to commitAdmission when overlay-admission-v1 is advertised.
MongoAdmissionStorage honors majority ACK, conditional spends, enlisted
indexes or a durable lookup/propagation outbox. Knex/SQL remains the default.
* fix(sdk): encode overlay discovery cache keys with stringifyBRC100
LookupResolver is a BRC-100 JSON boundary file, so the in-flight
discovery key cannot use JSON.stringify. stringifyBRC100 preserves the
same service-plus-limit tuple without changing cache cover semantics.
* fix(overlay): close S03 admission STEAK, history pin, and hydration gaps
Keep failed-topic STEAK keys and merkle applied metadata on the Engine
admission plan. Version history-update pins by generation and hydrate
Mongo output scripts and BEEF without unsafe integer coercion.
* fix(sonar): overlay S03 admission complexity and empty method
Extract Engine submit and admission plan validators, use localeCompare
and structuredClone, and throw from unimplemented Mongo BEEF updates.
* fix(sonar): evidence coordinator and chaintracks findings
Drop redundant optional-undefined on lookup evidence limits, extract
parseEvidence and Chaintracks header-retry helpers below S3776, and
rename coordinator catch params to error_.
* fix(sonar): overlay BASM protocol findings
Extract BASM reconcile helpers so cognitive complexity stays at or below 15,
prefer optional chaining on nullable anchors, and invert the advertised
response-length comparison.
* fix(sonar): overlay lookup discovery complexity and style
Extract query/session and host-tracking helpers so cognitive complexity
stays within Sonar's limit, and clear the remaining overlay-tools
findings without changing lookup behavior.
* fix(sonar): overlay mongo and admission findings
Brand StorageUint64, extract high-complexity helpers, and apply
Sonar-preferred optional chaining, regex, clone, and sort APIs
without changing payload identity or uint32 $toLong validators.
* fix(sonar): throw lookup abort instead of Promise.reject
S7746 on LookupResolver: prefer throw over return Promise.reject in the facilitator start chain.
* fix(wallet-toolbox): skip overlay identity when services are absent
discoverOverlayCertificates called getServices(), which throws on wallets
constructed without services and broke BRC-100 discoverBy* conformance.
Missing chain context now yields no identities instead of throwing.
* fix(ci): override js-yaml 3.15.2 for GHSA-2883
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
* fix(ci): override js-yaml 3.15.2 for GHSA-2883
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
* fix(ci): align sdk 2.5.0 health baseline and stack facts
Merge from main left baselines.json on 2.4.2 while package.json is 2.5.0.
* fix(ci): override js-yaml 3.15.2 for GHSA-2883
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
* fix(ci): override js-yaml 3.15.2 for GHSA-2883
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
* fix(ci): record js-yaml override removal rehearsal
Health requires overrideRemovalReview.retainedCount to match the live registry.
* fix(ci): record js-yaml override removal rehearsal
Health requires overrideRemovalReview.retainedCount to match the live registry.
* fix(ci): record js-yaml override removal rehearsal
Health requires overrideRemovalReview.retainedCount to match the live registry.
* fix(ci): record js-yaml override removal rehearsal
Health requires overrideRemovalReview.retainedCount to match the live registry.
* fix(ci): record js-yaml override removal rehearsal
Health requires overrideRemovalReview.retainedCount to match the live registry.
* fix(ci): ratchet override count for js-yaml 3.15.2
* fix(ci): ratchet override count for js-yaml 3.15.2
* fix(ci): ratchet override count for js-yaml 3.15.2
* fix(ci): ratchet override count for js-yaml 3.15.2
* fix(ci): ratchet override count for js-yaml 3.15.2
* fix(overlay): brand StorageUint64 at admission write sites
S6564 made StorageUint64 a unique branded string. Engine and Mongo
admission paths now validate decimal strings through asStorageUint64.
* fix(ci): raise SDK and wallet platform bundle budgets
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads
over the previous raw-size gates.
* fix(ci): raise SDK and wallet platform bundle budgets
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads
over the previous raw-size gates.
* fix(ci): raise SDK and wallet platform bundle budgets
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads
over the previous raw-size gates.
* fix(ci): raise SDK and wallet platform bundle budgets
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads
over the previous raw-size gates.
* fix(ci): raise SDK and wallet platform bundle budgets
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads
over the previous raw-size gates.
* fix(ci): raise remaining SDK-consumer bundle budgets
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the
evidence/discovery helper payload growth.
* fix(ci): raise remaining SDK-consumer bundle budgets
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the
evidence/discovery helper payload growth.
* fix(ci): raise remaining SDK-consumer bundle budgets
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the
evidence/discovery helper payload growth.
* fix(ci): raise remaining SDK-consumer bundle budgets
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the
evidence/discovery helper payload growth.
* fix(ci): raise remaining SDK-consumer bundle budgets
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the
evidence/discovery helper payload growth.
* fix(ci): raise wallet client esbuild brotli budget
* fix(ci): raise wallet client esbuild brotli budget
* fix(ci): raise wallet client esbuild brotli budget
* fix(ci): raise wallet client esbuild brotli budget
* fix(ci): raise wallet client esbuild brotli budget
* fix(ci): raise did-client UMD budget for SDK helper growth
* fix(ci): raise did-client UMD budget for SDK helper growth
* fix(ci): raise did-client UMD budget for SDK helper growth
* fix(ci): raise did-client UMD budget for SDK helper growth
* fix(ci): raise did-client UMD budget for SDK helper growth
* fix(ci): prettier browser budget JSON
* fix(ci): prettier browser budget JSON
* fix(ci): prettier browser budget JSON
* fix(ci): prettier browser budget JSON
* fix(ci): prettier browser budget JSON
* fix(ci): raise did-client esbuild budget
* fix(ci): raise did-client esbuild budget
* fix(ci): raise did-client esbuild budget
* fix(ci): raise did-client esbuild budget
* fix(ci): raise did-client esbuild budget
* fix(ci): raise did-client vite budget
* fix(ci): raise did-client vite budget
* fix(ci): raise did-client vite budget
* fix(ci): raise did-client vite budget
* fix(ci): raise did-client vite budget
* fix(ci): raise SDK vite and esbuild compressed budgets
* fix(ci): raise SDK vite and esbuild compressed budgets
* fix(ci): raise SDK vite and esbuild compressed budgets
* fix(ci): raise SDK vite and esbuild compressed budgets
* fix(ci): raise SDK vite and esbuild compressed budgets
* fix(wallet-toolbox): throw on overlay identity forceRefresh without services
Contact discovery still works without services. forceRefresh bypasses
contacts and still requires a chain tracker, matching the existing test.
* fix(overlay): retry Mongo write conflicts as transient admission errors
Concurrent snapshot spends can surface WriteConflict without the
TransientTransactionError label. Retry those so the loser observes spend-conflict.
* test(overlay): cover BASM remote limits for patch coverage
* fix(sonar): extract overlay chain-tracker guard from Wallet discovery
S3776: discoverOverlayCertificates was 17 after the forceRefresh services
check. The missing-services path is now requireOverlayChainTracker.
* test(sdk): cover overlay discovery helpers for patch coverage
* test(overlay): cover mongo storage helpers for patch coverage
* fix(sonar): use toHaveLength in LookupResolver discovery tests
* fix(overlay): retry Mongo write conflicts on claim as well as body
Concurrent snapshot spends can throw WriteConflict from findOneAndUpdate
outside the transaction body. Treat those as transient at the run loop.
* test(sdk): cover chain tracker and evidence helpers for patch coverage
* fix(overlay): retry admission commits on Mongo write conflicts
Concurrent snapshot spends still escaped the transaction-runner retry.
commitAdmission now retries WriteConflict up to eight times.
* fix(sonar): extract admission write-conflict retry helper
S3776: commitAdmission was 21 after the retry loop. The loop now lives in
runAdmissionWithWriteConflictRetry.
* test(overlay): cover mongo admission commit for patch coverage
* fix(sonar): use toHaveLength in Engine admission tests
* fix(overlay): reject unsafe Mongo GASP cursor values
* fix(ci): parent and finalize Codecov coverage reports
* fix(ci): parent and finalize Codecov coverage reports
* fix(ci): parent and finalize Codecov coverage reports
* fix(ci): parent and finalize Codecov coverage reports
* docs: reverify resource and TTN rollout gates
* docs: reverify resource and TTN rollout gates
* docs: reverify resource and TTN rollout gates
* docs: reverify resource and TTN rollout gates
* fix(ci): require HTTPS for Codecov polling
* fix(ci): require HTTPS for Codecov polling
* fix(ci): require HTTPS for Codecov polling
* fix(ci): require HTTPS for Codecov polling
* fix(ci): raise SDK esbuild raw browser budget
* test(sdk): cover invalid evidence intake limits
* test(overlay): cover uint64 payload length boundary
* test(overlay): cover admission output-index bound
* test(overlay): advertise the Go interop listener as explicit loopback
* fix overlay replay identity and evicted output reads
* fix(ci): remove duplicate workspace override
* fix(ci): synchronize admission lockfile overrides
* fix: make overlay admission runtime portable
* fix: make overlay admission runtime portable
* fix: avoid eager fetch dependency in broadcaster
* fix: avoid eager fetch dependency in broadcaster
* fix: defer unavailable fetch failure until send
* fix: defer unavailable fetch failure until send
* fix: defer unavailable fetch failure until send
* fix: defer unavailable fetch failure until send
* test: provide fetch spy target in node
* test: match complete admission acknowledgment
* chore(overlays): re-derive release metadata after the ordered merges
Refreshes the overlay package documentation dates for the four packages
this change bumps and regenerates the derived reference documents so the
stack facts and package API migration tables match the merged versions.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(overlays): re-derive release metadata after the ordered merges
Apply this PR's own SemVer impact on top of what the predecessor branch
leaves, and restore the cumulative release prose the branch had regressed.
- Versions: @bsv/overlay 2.5.0, @bsv/overlay-express 2.7.0,
@bsv/overlay-discovery-services 2.2.3, @bsv/overlay-topics 1.8.2, with
matching repository-health baselines and docs frontmatter.
- governance/package-release-notes.json: keep main's and the predecessor's
published versions and cumulative text, and append only this PR's BASM
sentences for the four overlay packages.
- docs/packages/sdk/bsv-sdk.md: make the Merkle-path candidate note
version-neutral so it no longer names a superseded release.
- Regenerate docs/reference/package-api-migrations.md, stack-facts.md and
service-operations.md.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(overlays): re-derive release metadata after the ordered merges
Bumps the overlay family's own version fields on top of what #517-#520
already carry: @bsv/overlay 2.6.0, @bsv/overlay-express 2.7.1,
@bsv/overlay-discovery-services 2.2.4, @bsv/overlay-topics 1.8.3.
No @bsv/sdk or @bsv/wallet-toolbox version bump is included; that
release decision stays with the maintainer.
Updates governance/repository-health/baselines.json's
publicPackageVersions for the four bumped packages, and regenerates
docs/reference/package-api-migrations.md and docs/reference/stack-facts.md
(pnpm docs:facts) plus each package's docs page frontmatter so they
match. governance/package-release-notes.json's cumulative summary and
migration prose for @bsv/overlay already carries this PR's own opt-in
Mongo AdmissionStorage adapter / Engine.submit() admission-commit
sentence, layered onto #519/#520's BASM prose during the predecessor
merge; no further edit is needed here.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(sdk): keep the evidence branch esbuild budget at its measured ceiling
The main integration merge raised the SDK esbuild raw ceiling to 600000,
a value that belongs to the lookup discovery change (#518). This branch
measured 590000; restore it so the diff carries only its own budget.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(sdk): scope lookup discovery docs and budget to this change
Restore the eleven ts2md reference pages for modules this change does
not touch to the incoming copies; the generator run had imported
unrelated drift that already exists on main. Keep the regenerated
overlay-tools page. Reinstate this change's own 600000 esbuild raw
ceiling after restacking on the corrected evidence branch.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(sdk): record the Merkle path offset correction in the release notes
The ordered-merge integration took the incoming @bsv/sdk release-note
entry verbatim and dropped this change's sentences. Append them to the
cumulative candidate text and regenerate the package migration table.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(sdk): reject cancelled lookups instead of answering empty
query() and queryDetailed() flatten query$ to its first snapshot. The
terminal gate only threw when hostCount was 0, and explicitly skipped
the throw for terminalReason 'cancelled', so a caller abort resolved as
{ type: 'output-list', outputs: [] }. At any host count an aborted
attempt was indistinguishable from an authoritative "this service has
no outputs" answer, which is exactly the completion-vs-empty confusion
queryDetailed exists to prevent.
Throw lookupAbortError() before the host-count gate whenever the
terminal snapshot reports 'cancelled'. query$ is unchanged and still
emits its terminalReason: 'cancelled' snapshot for progressive callers.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(sdk): keep discovery resource limits out of the no-hosts error
chargeLookupDiscoveryBytes can exhaust maxTotalBytes while SLAP trackers
are still being read, before any host is admitted. That leaves the
terminal snapshot with hostCount 0, terminalReason 'resource-limit' and
the limit recorded in limitsHit. queryDetailed()'s hostCount gate did not
look at terminalReason, so it threw the historical "No competent <network>
hosts found by the SLAP trackers" Error and discarded both fields.
Security-sensitive Promise callers could not tell "the trackers named no
hosts" from "this attempt ran out of its own byte budget".
Branch the gate on terminalReason: 'resource-limit' throws
LookupResourceLimitError carrying the first limit that was hit, while
'deadline' and a settled empty discovery keep the historical error and
message unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(overlay): keep SPV verification on historical-tx submissions
A merge on this branch widened the submit() SPV guard from
`mode !== 'historical-tx-no-spv'` to `mode === 'current-tx'`. That removed
`tx.verify(this.chainTracker)` from every `historical-tx` submission, not just
from the BASM admission path that needed it. `historical-tx` is a public
submission mode, so any caller could get an unverified transaction applied to
a topic.
Root cause: the guard was widened to work around a symptom. BASM admission
proves inclusion independently (`chainTracker.isValidRootForHeight` plus the
canonical-header binding) and must not re-apply `MerklePath.verify`'s coinbase
100-block spendability rule, but the correct lever for that is the existing
`historical-tx-no-spv` mode on the BASM call site, not a relaxed global guard.
Restores the original guard and switches the BASM admission in
`fetchBASMMissingTransactions` to `historical-tx-no-spv`. The other effects of
that mode are correct here: propagation is skipped (already the case for
`historical-tx`), and although the broadcast block is entered, every
BASM-admitted transaction carries its extracted Merkle path, so
`broadcastAcceptedSubmission` synthesises a local success and never reaches the
network. Topic managers now observe `historical-tx-no-spv` for BASM
admissions; no bundled manager branches on the mode.
Tests: the coinbase fixture no longer mocks `Engine.submit`, so the young
offset-0 proof is admitted through the production path, and a new test proves a
GASP-style `historical-tx` submission with an invalid proof is still rejected.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(sdk): refuse redirects on overlay lookup and SLAP discovery requests
performLookupRequest built its RequestInit without a redirect field, so
fetch defaulted to 'follow'. normalizeLookupHost and the https: guard in
lookup() validate the advertised URL only; neither runs again on the
final URL. A SLAP-advertised HTTPS host could therefore answer the
lookup POST with a 307/308 and have the serialized
stringifyBRC100({ service, query }) body re-sent to http:, loopback or a
link-local address. This PR queries every eligible advertised host, so
the hop sat on the hot path for untrusted SLAP domains, and the SLAP
tracker discovery requests reach the network through this same method.
Set redirect: 'error' so the transport fails closed. fetch rejects, and
the existing failure path records the advertised host as an ordinary
availability failure instead of crashing the query or following the hop.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): gate Go interop solely on BASM_GO_OVERLAY_SERVICES
`resolveGoWorktree()` tried the `BASM_GO_OVERLAY_SERVICES` environment variable
and then fell back to the literal path
`/Users/personal/git/go/worktrees/go-overlay-services-basm`. That fallback made
the suite run on exactly one workstation and `describe.skip` everywhere else,
including CI, so the B01 protocol was never actually spoken by any job that
claimed to cover it, and the skip was silent.
Root cause: the checkout location was treated as a discovery problem with a
convenient default instead of an explicit input. Resolution is now the
environment variable alone. When it is unset the suite is skipped under a title
that names the variable; when it is set but does not resolve to a
go-overlay-services checkout the resolver throws, so a job configured for Go
interop fails closed rather than quietly skipping.
Tests: a new BASMGoInteropGating suite asserts the gate from the interop
source, which is the only machine-independent way to catch this class of
regression - a behavioural assertion agrees with a workstation fallback on
every machine that lacks that path, which is precisely where the fallback
hides. The fixtures README now states that Go interop does not run by default
and records the fixture provenance as a repository-relative path.
Verified locally: with
BASM_GO_OVERLAY_SERVICES=/Users/personal/git/go/worktrees/go-overlay-services-basm
the interop suite builds the Go host and passes (1/1); with a non-checkout path
the suite fails to run with the explanatory error; unset, it skips.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(overlay-mongo): serialize pin re-add expiry with GC reclaim
addReference's idempotent re-add branch only re-validated the payload
was still ready; it never touched the existing (ownerKind, ownerId,
slot) pin row's expiresAt. Extending a pin therefore silently kept the
stale expiry, and claimGarbage counted an already-expired pin as
not-live via $expr {$gt:['$expiresAt','$$NOW']} but never deleted that
expired pin document. The unique slot row survived expiry and kept
short-circuiting future addReference calls on that slot, so a caller
could be told content stayed pinned while GC moved the payload
ready -> deleting underneath it.
Add refreshPinExpiry: called after the existing {_id: payloadId,
state: 'ready'} CAS matches (so it stays serialized against a
concurrent GC claim in the same transaction), it reactivates an
expired pin with the caller's new expiry, allows extending a still-
live pin to a later expiry, and rejects an attempt to shorten a live
pin's expiry rather than silently ignoring it. claimGarbage now
deletes every expired pin reference row for the payload in the same
transaction as its ready -> deleting claim, so a stale slot cannot
outlive the payload it named.
Document the re-add/extend/reactivate/shorten rule in
specs/overlay/mongo-v1.md next to the existing pin semantics.
Regression tests (mongodb-memory-server replica fixture) added to
src/__tests/mongo/MongoPayloadStore.test.ts:
- re-adding a pin slot extends its expiry but never shortens it
silently
- re-adding an expired pin slot reactivates it and makes the payload
live again
- claimGarbage deletes the expired pin reference row inside the same
transaction as its claim
All three were confirmed to fail against the pre-fix source for the
reason described above (verified by temporarily reverting this file
and re-running the suite) before the fix was applied.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(overlay): record the BASM admission submission mode
Follow-up to 7584d4e8c, which moved BASM admission from the public
`historical-tx` submission mode to `historical-tx-no-spv` so that restoring the
SPV guard does not re-apply MerklePath.verify's coinbase 100-block spendability
rule to an independently proven admission.
The submission mode is passed through to `TopicManager.identifyAdmissibleOutputs`,
so it is observable by every topic manager that branches on it. The release
prose described the inclusion rule but not the mode a manager now sees, which
left implementors without the one externally visible consequence of the change.
Adds that sentence to the @bsv/overlay release note and the package CHANGELOG
entry, and regenerates docs/reference/package-api-migrations.md from the
governance source. No code or behaviour change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay-mongo): cover the too-small declared length reclaim path
The reviewer asked for a regression test on the reclaim path taken when
publish fails because the caller declared a too-small byteLength:
consumePayloadChunk rejects the stream once it exceeds the declared
length, abandonUpload marks the payload row 'deleted', and a retry
with the correct digest and length must reclaim that row through
tryClaimUpload (which already recomputes byteLength from the new input
when the prior state is 'deleted') and finish 'ready' with the
corrected byteLength stored.
This path was already implemented and covered indirectly (a sibling
test manually sets state: 'deleted' with a wrong byteLength via direct
DB mutation), but there was no test driving the actual publish/fail/
retry sequence end to end. This is coverage-only: the reclaim logic in
tryClaimUpload predates this change and needed no fix, so there is no
meaningful pre-fix state to fail against in this worktree; the new
test passes against the current code, which is the expected outcome
for confirming existing, correct behavior.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(sdk): re-queue displaced same-txid candidates instead of dropping them
TransactionEvidenceCoordinator.run() shifts the next candidate off a job's
queue before calling attempt(). If every concurrency slot is already in use
(including slots still reserved by non-abortable, already-finished attempts;
see the "does not free a non-abortable backend slot..." test), attempt()
threw a plain TransactionEvidenceError('limit') and called the caller's
`settled` callback. run()'s catch treated that exactly like a genuine
verification failure: the candidate was gone (already shift()ed) and never
pushed back, so a same-txid alternate candidate that had already been
admitted to the job could be silently discarded without ever being tried,
even though it was perfectly valid.
Fix: attempt() now throws a dedicated ConcurrencyLimitSignal when the
operation never started, and does not invoke `settled` (nothing was
consumed, so no byte/candidate bookkeeping should be released). run()'s
candidate loop catches that signal specifically, unshifts the candidate
back onto job.candidates, marks the job not-running, and returns without
looping or retrying itself. Re-admission is driven solely by pump() being
invoked again when some other active attempt settles, so this cannot spin
or busy-wait for a slot.
Added a regression test that builds a genuine concurrency race: a
non-abortable "ghost" attempt (cancelled while its script verification is
still in flight) holds one of two slots, a job with two same-txid
candidates (one invalid, one valid) takes the other, and a third waiting
job is admitted into the slot freed by the first candidate's failure before
the job's own retry can reclaim it. Before the fix this made the valid
alternate candidate reject with 'limit'; the test fails for that reason on
the old code and passes with the fix.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(wallet-toolbox): implement BHServiceClient.findChainTipHash
Every TransactionEvidenceCoordinator attempt calls
ChaintracksChainTracker.getVerificationContextToken(), which requires
chaintracks.findChainTipHash(). BHServiceClient implemented that method
as `throw new Error('Not implemented')` even though it already exposes a
working findChainTipHeader() against /api/v1/chain/tip/longest. Any wallet
using a BHServiceClient as options.chaintracks, or as a LocalChainTracker
participating source, broke on every verification attempt.
Fix: implement findChainTipHash() by delegating to the existing
findChainTipHeader() and returning its hash, matching the same pattern
GoChaintracksServiceClient already uses for the same interface method.
No other ChaintracksClientApi contract surface changes.
Added a regression test on BHServiceClient confirming findChainTipHash()
resolves to findChainTipHeader()'s hash instead of throwing, and a
regression test on ChaintracksChainTracker confirming
getVerificationContextToken() succeeds end-to-end when backed by a
BHServiceClient. Both fail with "Not implemented" on the old code and
pass with the fix.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: changelog entries for F1/F2 review-finding fixes
Document the TransactionEvidenceCoordinator candidate-requeue fix and the
BHServiceClient.findChainTipHash implementation in their packages'
CHANGELOG.md, matching this repo's convention of recording behavior-
affecting fixes there alongside the commit history.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* style: restore newline dropped by the restack conflict resolution
Whitespace only; the merged release-notes content is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(overlay): exclude dupe topics from Mongo admission commit (F1)
buildOverlayAdmissionPlan's `accepted` filter treated an already-applied
(isDupe) topic the same as a genuinely new admission, so it still received
an identity entry, a decision, and a place in the commit. Any retry that
resubmits an already-applied topic under a different admission operation —
OverlayGASPStorage.finalizeGraph submitting 'historical-tx-no-spv' after a
live admit, a live resubmit of a historically synced tx, or a multi-topic
retry that mixes an applied topic with a new one — produces a plan whose
operationId differs from the one recorded when that topic was first
applied. MongoAdmissionStorage.assertAppliedAvailable rejects any applied
row whose admissionId belongs to a different operation, so the retry threw
"Overlay admission rejected: invalid-plan" (or 'digest-mismatch') instead of
being idempotent.
Fix: exclude isDupe topics from the plan's identity/decisions via a new
selectNewAdmissionTopics() helper, shared by EngineAdmission.ts and
Engine.ts. When every remaining topic is a dupe or a failure,
Engine.acknowledgeOverlayAdmission now returns the in-memory STEAK (already
computed from validation) without building or committing a plan at all. On
a mixed set, only the new topics are committed; the STEAK returned still
reports each dupe topic as accepted-with-nothing-new, matching what the
classic (non-admission) storage path already returns for a dupe.
Updated one existing EngineAdmission unit test that had encoded the old
(buggy) contract directly — asserting that an isDupe-only validation still
produced a decision with merkle metadata — to use a real admission instead,
since a dupe-only plan is no longer a supported call shape (documented by a
new test asserting it throws).
Tests: new Engine-level cases for all-dupe, mixed dupe/new, and
resubmit-after-historical, plus a real MongoAdmissionStorage-backed
regression (via Engine + MongoOverlayStorage) that reproduces the exact
multi-topic "invalid-plan" rejection this fixes. New EngineAdmission unit
tests cover buildOverlayAdmissionPlan's dupe exclusion directly. Full
@bsv/overlay suite (mock + mongo/**) passes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(overlay-mongo): never surface an evicted output as live (F2)
MongoOverlayStorage.toOutput mapped `spent: document.state === 'spent'`,
so an evicted document (state: 'evicted', kept for audit/history rather
than deleted) would map to `spent: false` — a live, unspent output — if it
ever reached this function. findOutput and findOutputsForTransaction
already excluded `evicted` at the query level, but that meant the
"evicted is never live" invariant lived only in two separate query
filters rather than in the one function actually responsible for shaping
an Output, so a future read path that forgot the filter would silently
resurface deleted UTXOs. toOutput also hard-coded outputsConsumed: [] and
consumedBy: [], so the consumption edges written during an admission
commit (or via updateConsumedBy on the classic path) were invisible to
every reader.
Fix: toOutput now returns `Output | null`, returning null for any
`state: 'evicted'` document; all three read paths (findOutput,
findOutputsForTransaction, findUTXOsForTopic) go through a shared
toOutputs() helper that drops null mappings the same way, so the
invariant holds regardless of each query's own filter. toOutput now also
hydrates outputsConsumed/consumedBy by querying the consumptionEdges
collection persisted by MongoAdmissionStorage.insertEdge and
MongoOverlayStorage.updateConsumedBy.
Tests: a direct test of toOutput (the two current public read paths
already filter evicted at the query level, so this is the only way to
observe the mapping bug fail on current code) confirms an evicted
document is never mapped to a live Output; a findOutput/
findOutputsForTransaction test confirms outputsConsumed/consumedBy are
hydrated from persisted edges in both directions. Full @bsv/overlay
mongo/** suite passes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(sdk): state why a cached evidence recheck cannot be displaced
The concurrency-limit signal is handled only in the candidate loop. The
cached-positive recheck does not need the same handling: run() is
reached solely from pump(), which checks the attempt limit synchronously
before starting the job, and no await separates that check from the
recheck attempt. Record the invariant next to the call so a later
refactor of pump() or run() revisits it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(wallet): set platform budgets from measured evidence-branch bundles
The ceilings carried on this branch were authored before main grew
(#534, #547), so the merged wallet bundles exceeded them and CI stopped
at the first over-budget dimension. Measured from the exact packed
dependency graph at 83c067098 (JS bundle sizes verified byte-identical
between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux
ratio recorded in #547):
client Vite 1,759,717 raw / 416,284 gzip / 324,287 brotli
client esbuild 1,372,320 raw / 378,741 gzip / 302,937 brotli
mobile Metro 1,811,581 raw / 461,355 gzip / 355,871 brotli
mobile Hermes 3,688,871 raw / 1,502,972 gzip / 1,166,821 brotli
Only the dimensions that exceeded are raised, with about 0.25% headroom
(1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(budgets): set ceilings from measured lookup discovery bundles
Bounded discovery, the host queue and the streaming response reader grow
every bundle that embeds @bsv/sdk. Measured from the exact packed
dependency graph at the top of the ordered overlay wave (max of 5eef0fc67
and 907739402; JS bundle sizes verified byte-identical between macOS and
hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio from #547):
@bsv/sdk UMD 591,705 raw / 168,531 gzip / 140,485 brotli
@bsv/simple Vite 752,122 raw / 182,787 gzip / 150,699 brotli
wallet client Vite 1,782,018 raw / 421,917 gzip / 328,724 brotli
wallet client esbuild 1,389,928 raw / 383,620 gzip / 306,783 brotli
wallet mobile Metro 1,834,234 raw / 466,804 gzip / 360,591 brotli
wallet mobile Hermes 3,725,120 raw / 1,520,644 gzip / 1,181,384 brotli
Only the dimensions that exceeded are raised, with about 0.25% headroom
(1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance).
Later branches in the wave move these bundles by under 150 bytes, so
they inherit these ceilings unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): name the historical-tx SPV test for what it covers
GASP finalization already submits with historical-tx-no-spv; its
historical-tx calls are dry-run admissibility checks. The restored guard
protects public Engine.submit callers, so say that in the test title.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): exclude src/__tests support modules from coverage
The repository's patch-coverage gate (scripts/patch-coverage.mjs,
TEST_PATH pattern /(?:^|\/)__tests(?:__)?(?:\/|$)/) treats both
`__tests/` and `__tests__/` directories as test code. This package's
jest collectCoverageFrom only excluded `__tests__/**`, so the
non-`.test.ts` support modules this PR adds under src/__tests/
(admission/ReferenceAdmissionStorage.ts,
admission/AdmissionStorageContract.ts, mongo/MongoCommitResponseProxy.ts,
mongo/MongoReplicaFixture.ts, ...) were instrumented and reported to
Codecov as production code, understating the patch-coverage denominator
correction the gate already applies elsewhere. Add '!src/**/__tests/**'
so this package's own coverage collection matches the gate's definition
of test code.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): share the BASM reconciliation peer fixture
Move the fake-peer fixture out of BASMReconciliation.test.ts so further
fail-closed suites can drive the same engine, storage, chain tracker and
fetch mock without duplicating them. No test behaviour changes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): pin BASM_UNSUPPORTED provider guards
Each BASM read endpoint fails closed with a TypeError carrying
code 'BASM_UNSUPPORTED' when the storage backend does not implement the
optional method behind it, and the capability check runs before the
argument-shape guard on compound Merkle paths.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): cover BASM tip-comparison and error reporting paths
Exercises reconcileBASMWithPeer's not-ahead branch end to end: two empty
topics match without a range request, an equal matching tip is only
accepted after its local anchor is re-verified against the canonical
header, a higher local tip reports divergence, and a tip whose anchor is
missing, unsupported, TAC-mismatched or non-canonical fails closed.
Also pins the report surface: "matched" once the page catches the local
tip up, a stringified non-Error failure with no invented errorCode, and
BASM_UNSUPPORTED surfacing through startBASMSync.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): cover BASM compound-proof position binding
Adds the canonical-position cases the reconciler fails closed on: a
phantom duplicate at the right edge of an odd-width block is accepted but
the same node is rejected once the canonical width is even, a BUMP that
omits an admitted leaf is rejected before any raw transaction is fetched,
a non-canonical root is rejected by the chain tracker, canonical-count
assurance is recorded even when nothing needs fetching, and an anchor
that vanishes from the peer blocks admission.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): cover BASM forward-page prefix and assurance reporting
Drives a multi-block fake peer to pin the anchor-page contract: a page
truncated short of its requested target and a page that skips the height
after the local tip are both rejected, a peer many pages ahead is
followed one capped page at a time without re-downloading anchors that
already match locally, and a page mixing canonical-count and
encoded-offset blocks reports the weaker assurance for the whole attempt.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(sdk): pin HTTPS lookup facilitator transport bounds
Cover the cancellation and resource-limit branches of
HTTPSOverlayLookupFacilitator: a pre-aborted caller signal short-circuits
before any fetch, an abort racing a non-ok response reports cancellation and
drains the body, JSON and octet-stream output counts are rejected above the
caller's budget (including a negative varint count), atomic BEEF extraction
stops once the extracted bytes outrun the response budget, and an in-flight
decode observes a mid-stream cancellation.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* style(overlay): prettier-format the new BASM test suites
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(sdk): pin lookup query resource, cancellation and host-policy bounds
Exercise the uncovered fail-closed branches of LookupResolver.query$:
aggregation stopping at maxOutputs across hosts, the evidenceLimits shorthand
defaulting its byte budget, the bounded candidate scan (scan window, malformed
entries, maxHosts), per-source quotas for additional hosts, SLAP discovery
refusing bytes that would breach maxTotalBytes, byte reports arriving after
cancellation, cancellation dropping an in-flight peer answer and skipping
queued hosts, idempotent double cancellation, the concurrent-query ceiling,
over-budget answers being dropped without blaming the host, the maxTrackers
budget, advertisement-map eviction, and a broader host cache surviving a
tighter rediscovery.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): cover MongoAdmissionPlan STEAK-binding fail-closed branches
Adds cases for isBoundSteak rejecting a non-object STEAK entry, an entry
missing outputsToAdmit, a STEAK record that omits the plan's own decision
topic, and an output index that fails to parse while binding STEAK; also
covers validateAdmissionPlan rejecting a non-canonical expected-history
fence. Two branches in samePayload's optional chaining are confirmed
unreachable (they require a payload ref with an undefined digest, which
the type system and every real caller already rule out).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): cover EngineAdmission STEAK/coin-recovery branches and Engine's no-callback path
EngineAdmission.test.ts: covers the top-level STEAK pass carrying a
caller-supplied coinsRemoved through unchanged for a topic the accepted
loop never re-derives, and classifyCoins recovering a previous txid from
sourceTransaction.id() when sourceTXID is absent.
Engine.admission.test.ts: covers acknowledgeOverlayAdmission's early
return when no topic was accepted and no onSteakReady callback was
supplied, so the optional-callback branch is exercised without throwing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): cover MongoAdmissionStorage retry/pending orchestration via injected runner
New file, mocked MongoTransactionRunner, no real MongoDB: covers
isMongoWriteConflict's non-object/null guard, waitForPending's 50-poll
timeout falling through to a fresh commit attempt, and non-Mongo/non-Error
rejections from the transaction body propagating unclassified rather than
being retried or swallowed. Confirms the write-conflict loop's post-loop
fallback (`return { state: 'rejected', code: 'spend-conflict' }`) is dead
code: the loop's own bound guarantees it always returns/throws by the 8th
attempt, so that line can never execute.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): cover MongoAdmissionStorage fail-closed and fencing branches
Adds real-replica-set coverage for: a fence conflict rejecting a stale
expected history; re-admitting the same tx/topic under a different
operation id (fenced off via assertAppliedAvailable, exercising the
idempotent-spend return in applySpend along the way); a real eviction
succeeding; output/applied-history insert collisions caught as
invalid-plan, plus their non-duplicate-key-error fallthroughs (via a
namespace-scoped injected write failure); omitting firstSeenHeight;
upsertTransaction's raw/manifest presence branches; an unrecognized
payload kind rejected while pinning; a payload-store failure while pinning
classified as payload-not-ready, and an unrelated one left unclassified;
claimOutbox tolerating a since-deleted pinned payload; reconcileAdmission
delegating to (and reusing) a scoped peer; applyHandoff's happy path,
fence mismatch, and missing-lease paths; and two best-effort concurrent
races (generation bootstrap, outbox event id) that each still resolve to
exactly one winner and one clean rejection.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): cover MongoOverlayStorage read/hydration branches
Covers: reusing a caller-supplied payload store instead of constructing
one; insertOutput recording a spent output from the start; findOutput
with includeBEEF leaving beef unset when no transaction row exists, and
when the transaction row names a payload that is no longer ready;
readScript returning an empty script for a (schema-legacy, simulated via
bypassDocumentValidation) output document with no scriptPayloadId; and
readBeef hydrating a real merkle path pinned by an admission commit into
an atomic BEEF, then falling back to the plain BEEF when that merkle
payload is later removed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): cover MongoPayloadStore read()/readBytes() in a new file
New file, scoped to read()/readBytes() only per this PR's worklist (a
concurrent branch owns the rest of MongoPayloadStore.ts's coverage, in
its own MongoPayloadStore.test.ts, so this stays a separate file to keep
the eventual merge conflict-free). Covers: read() on missing content;
read() rejecting a negative offset, a negative length, and an
offset+length past the end; read() rejecting a range whose end would
exceed a safe JS integer; readBytes() downloading GridFS-spilled content
across chunks (including a partial, mid-file range); and readBytes()
failing closed on a missing GridFS file and on a ready record with
neither inline data nor a file id.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): fix scope mismatch and range invariant, add GC-tolerant pin coverage
Two follow-ups to the prior MongoAdmissionStorage commit: the
unrecognized-payload-kind test kept the output's script offset/byteLength
pointed at the original (25-byte) script, so isInvalidOutput's own range
check rejected the plan before pin() was ever reached -- fixed to match
the substituted payload's declared byteLength, which now actually reaches
and exercises pin()'s isPayloadKind guard.
Also adds coverage for replaceHistoryPin tolerating a previous
history-update pin whose payload document has since been garbage
collected while its reference row remains, skipping the release instead
of dereferencing a missing document.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): pin MongoPayloadStore fail-closed and fencing behavior
Adds real jest tests for previously-uncovered MongoPayloadStore branches:
pin re-add no-op, claimGarbage/finishGarbage edge cases (never-published
digest, afterDeleteClaim hook, ownership-metadata drift), lease-stolen
fencing before the ready CAS and before GridFS staging, wrong explicit
txid on both the fresh-upload and already-ready paths, GridFS
crash-boundary hooks, abandonUpload's guard against deleting a
concurrently-published file, digest-mismatch-after-spillover cleanup,
stale inline-upload recovery, an expired crashed reservation reclaiming
and retiring its orphaned GridFS file, and the abort-signal fallback
messages used when a signal's `reason` is unset. Also stresses a
many-way concurrent first-time publish of the same digest and a
corrupted-in-place staged GridFS chunk to exercise the duplicate-key
race and post-upload verification failure paths.
Uncovered lines for this file drop from 57 to 21 (lcov, line+branch).
Remaining lines are either dead code given existing invariants (the
per-chunk maxPayloadBytes check, verifyGridFs's non-byte-chunk guard,
the "stream writer missing" guard) or depend on timing-sensitive races
with no exposed test seam (recoverStaleUpload's current-state guard,
the exact instant a GridFS write signals backpressorem).
A test also surfaced a real defect, reported separately: finishGarbage's
FileNotFound-swallow regex never matches the mongodb driver's actual
"File not found for id X" message, so a benign already-deleted-file
race during cleanup always rethrows instead of being swallowed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): pin MongoTransactionRunner claim races and reconcile edges
Adds real jest tests for previously-uncovered MongoTransactionRunner
branches: many concurrent fresh claims on the same operation id (forces
the duplicate-key insert race and the "lost the claim" observer path),
a claim-row insert failure whose error code is not a duplicate key
(must propagate, not be swallowed), reconcile() with no attemptId
against a never-created operation (the "unlocated" fallback), reconcile()
with a mismatched attemptId against a genuinely pending claim, and
commit failures whose deadline elapses mid-retry-loop (the
budget-aborted break, verified via a real blocked commitTransaction).
Uncovered lines for this file drop from 18 to 9 (lcov, line+branch).
Remaining lines depend on races or invariants not reachable from the
public API: Budget's manual deadline check is shadowed by the earlier
throwIfAborted() once the matching timer fires; abort()'s "already out
of transaction" branch has no path that reaches it before the catch
that calls it; the post-insert re-read uses a majority/primary read
right after an acknowledged write, so it can't observe a miss without
breaking the replica set's own consistency guarantee; and the
ownership-fencing mismatches and the "operation row disappeared"
branches require deleting a submission-operation row, which nothing in
this class ever does.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(overlay): pin MongoSchema bootstrap idempotency and failure paths
Adds real jest tests for previously-uncovered MongoSchema branches:
concurrent bootstraps of a fresh database (races the NamespaceExists
and duplicate-ledger swallow paths in createCollectionIfMissing,
ensureGridFs, and ensureLedger), a concurrent re-bootstrap after
dropping just the GridFS bucket collections, an existing collection
whose validator matches but omits an explicit collation (the
collation-compatible-by-omission branch), an index-creation failure
whose error code is not one of the benign conflict codes, and a ledger
insert failure whose error code is not a duplicate key (both must
propagate rather than being swallowed).
Uncovered lines for this file drop from 11 to 10 deterministically; the
concurrent-bootstrap races additionally land on the NamespaceExists
swallow branches in some runs but not every run, since they depend on
genuine scheduling of concurrent createCollection calls. The remaining
lines are dead code given the surrounding invariants (decodeMongoUint64's
re-encode check, the unused `string()` default length and
`allowAdditional` option on module-private helpers no call site ever
exercises) or require racing a second writer against
transactionalProbe's own read-then-CAS window, which was not attempted.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(overlay-mongo): tolerate an already-deleted GridFS file in finishGarbage
finishGarbage is documented as safe to retry, but it only swallowed
errors matching /FileNotFound/. The MongoDB driver reports a missing file
as MongoRuntimeError "File not found for id <id>", which never matched,
so a finisher that lost the race to a concurrent retry rethrew and left
the payload row in 'deleting'. Match the driver's actual message.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(overlay): reconcile BASM heights where the topic admitted nothing
BRC-136 defines an empty topic at a height as k = 0 with R = 32 zero
bytes, and rebuildTopicAnchorChain anchors every height that way. The
reconciler accepted such an anchor and then requested a compound Merkle
path for an empty txid list, which BASMRemote rejects ("At least one
BASM proof txid is required"). Any node behind a peer aborted at the
first height the topic admitted nothing, which is most heights.
After the root and count check, an empty admitted list has nothing to
bind or fetch: count the height as checked and skip the proof and raw
transaction round trip. It diverges only when this node admitted
transactions at that height. A zero-count anchor with a non-zero root is
still rejected.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(overlay): keep admission host methods bound to their storage
getOverlayAdmissionHost copied publishAdmissionPayload,
enlistedIndexTargets and getHistoryFence off the storage as detached
function references. Storage adapters are class instances, so inside
MongoOverlayStorage.publishAdmissionPayload `this` became the plain host
object and `this.payloads.publish` threw a TypeError: the Engine
admission path crashed on the first payload with the real Mongo adapter.
Every test reaching that path had mocked the method with jest.fn(),
which ignores its receiver.
Bind each optional method to the storage. Cover it with a class-based
unit test and a real replica-set test that publishes through the host
object the Engine uses; both failed before the change. The existing
helper test asserted function identity, which is the defect, and now
asserts that calls are forwarded with the storage as receiver.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overlay reliability program
This change is one slice of Make Overlays Great Again.
Do not merge until the stacked TypeScript and Go overlay branches have been proven together locally. Isolated CI green is not integration acceptance. Keep this PR draft. No npm publication, live database migration, or Mongo/BASM default activation.
C05 (verified progressive API) is parked. C02 (ts-stack#517) and C04 (ts-stack#518) both edit
LookupResolver.tsand must not merge independently.Program and scope
e82b77eeabfaf78e1a5b60f3367e56dc00fa828bImpact
Affected packages:
@bsv/sdkVerification
Security and dependencies
Release and operations
Notes for reviewers
Sibling C02/C03 PR also edits
LookupResolver.ts. Merge conflict expected. C05 stays parked.query()still throws when a deadline admits no host;query$remains an unverified transport aggregate.