Conversation
…duling Subscribe to SLAP tracker updates during an active query so later advertised hosts can join a fair, concurrency-bounded queue. Read lookup bodies incrementally with honest resource-limit accounting, keep trackers as routing sources, and leave raw query$ unverified while exposing the C02 onEvidence intake seam. Preserves 2s/5s delays, reputation/backoff, query/query$/freeform APIs, and existing CORS/public lookup request headers.
Add replica-set schema bootstrap, content-addressed payload publication, transactionally guarded references/GC, and majority-commit outcome helpers behind an optional mongodb peer. This is not an AdmissionStorage adapter and does not change Engine or Knex defaults.
Preserve per-entry script results so a rejected sibling cannot poison a shared ancestor. LocalChainTracker tokens use participating sources only and fail closed on missing identity. Reset ownership is rechecked after dispose before destructive hooks. Built-in remote ChainTracks clients advertise reorg-event capability explicitly. Services.getChainTracker publishes one coalesced wrapper before yielding.
Validate untrusted BASM peer JSON, bind remote anchors to local canonical headers, and independently check admitted-list block indices before historical admission. Keep automatic BASM sync disabled and leave durable recovery jobs for B02. Include independent BRC-136 vector checks and localhost TS client tests against the Go five-method read/serving surface.
BHServiceClient no longer caches the queried merkle root. Validity is decided from a freshly read header, so a false result cannot invert on retry and a reorg cannot reuse a stale positive.
Store discovery bounds on the SLAP host cache so a tighter-limit query cannot freeze a truncated host set for a later broader attempt. Throw the historical no-competent-hosts error from query()/queryDetailed when a deadline expires before any host is admitted; query$ still emits the deadline snapshot.
Collection validators used lexicographic $lte on unpadded outputIndex strings, which rejected legal indexes such as "9". Bound uint32 fields with $toLong on outputs, consumption edges, GASP nodes, and SHIP/SLAP.
Verify claimed admitted-list block indices against the compound Merkle path even when every remote txid is already local. Check inclusion with the chain-tracker root at the claimed height instead of MerklePath.verify, which also enforces coinbase 100-block spendability.
Wire Engine.submit to commitAdmission when overlay-admission-v1 is advertised. MongoAdmissionStorage honors majority ACK, conditional spends, enlisted indexes or a durable lookup/propagation outbox. Knex/SQL remains the default.
LookupResolver is a BRC-100 JSON boundary file, so the in-flight discovery key cannot use JSON.stringify. stringifyBRC100 preserves the same service-plus-limit tuple without changing cache cover semantics.
Keep failed-topic STEAK keys and merkle applied metadata on the Engine admission plan. Version history-update pins by generation and hydrate Mongo output scripts and BEEF without unsafe integer coercion.
Extract Engine submit and admission plan validators, use localeCompare and structuredClone, and throw from unimplemented Mongo BEEF updates.
Drop redundant optional-undefined on lookup evidence limits, extract parseEvidence and Chaintracks header-retry helpers below S3776, and rename coordinator catch params to error_.
Extract BASM reconcile helpers so cognitive complexity stays at or below 15, prefer optional chaining on nullable anchors, and invert the advertised response-length comparison.
…e-c02 # Conflicts: # docs/reference/package-api-migrations.md # governance/package-release-notes.json
…ardening # Conflicts: # docs/reference/package-api-migrations.md # docs/reference/stack-facts.md # governance/package-release-notes.json # governance/repository-health/baselines.json
Extract query/session and host-tracking helpers so cognitive complexity stays within Sonar's limit, and clear the remaining overlay-tools findings without changing lookup behavior.
Brand StorageUint64, extract high-complexity helpers, and apply Sonar-preferred optional chaining, regex, clone, and sort APIs without changing payload identity or uint32 $toLong validators.
# Conflicts: # docs/reference/package-api-migrations.md # docs/reference/stack-facts.md # governance/repository-health/baselines.json
S7746 on LookupResolver: prefer throw over return Promise.reject in the facilitator start chain.
discoverOverlayCertificates called getServices(), which throws on wallets constructed without services and broke BRC-100 discoverBy* conformance. Missing chain context now yields no identities instead of throwing.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Merge from main left baselines.json on 2.4.2 while package.json is 2.5.0.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
The repository's patch-coverage gate (scripts/patch-coverage.mjs, TEST_PATH pattern /(?:^|\/)__tests(?:__)?(?:\/|$)/) treats both `__tests/` and `__tests__/` directories as test code. This package's jest collectCoverageFrom only excluded `__tests__/**`, so the non-`.test.ts` support modules this PR adds under src/__tests/ (admission/ReferenceAdmissionStorage.ts, admission/AdmissionStorageContract.ts, mongo/MongoCommitResponseProxy.ts, mongo/MongoReplicaFixture.ts, ...) were instrumented and reported to Codecov as production code, understating the patch-coverage denominator correction the gate already applies elsewhere. Add '!src/**/__tests/**' so this package's own coverage collection matches the gate's definition of test code. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Move the fake-peer fixture out of BASMReconciliation.test.ts so further fail-closed suites can drive the same engine, storage, chain tracker and fetch mock without duplicating them. No test behaviour changes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Each BASM read endpoint fails closed with a TypeError carrying code 'BASM_UNSUPPORTED' when the storage backend does not implement the optional method behind it, and the capability check runs before the argument-shape guard on compound Merkle paths. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Exercises reconcileBASMWithPeer's not-ahead branch end to end: two empty topics match without a range request, an equal matching tip is only accepted after its local anchor is re-verified against the canonical header, a higher local tip reports divergence, and a tip whose anchor is missing, unsupported, TAC-mismatched or non-canonical fails closed. Also pins the report surface: "matched" once the page catches the local tip up, a stringified non-Error failure with no invented errorCode, and BASM_UNSUPPORTED surfacing through startBASMSync. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds the canonical-position cases the reconciler fails closed on: a phantom duplicate at the right edge of an odd-width block is accepted but the same node is rejected once the canonical width is even, a BUMP that omits an admitted leaf is rejected before any raw transaction is fetched, a non-canonical root is rejected by the chain tracker, canonical-count assurance is recorded even when nothing needs fetching, and an anchor that vanishes from the peer blocks admission. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Drives a multi-block fake peer to pin the anchor-page contract: a page truncated short of its requested target and a page that skips the height after the local tip are both rejected, a peer many pages ahead is followed one capped page at a time without re-downloading anchors that already match locally, and a page mixing canonical-count and encoded-offset blocks reports the weaker assurance for the whole attempt. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Cover the cancellation and resource-limit branches of HTTPSOverlayLookupFacilitator: a pre-aborted caller signal short-circuits before any fetch, an abort racing a non-ok response reports cancellation and drains the body, JSON and octet-stream output counts are rejected above the caller's budget (including a negative varint count), atomic BEEF extraction stops once the extracted bytes outrun the response budget, and an in-flight decode observes a mid-stream cancellation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…unds Exercise the uncovered fail-closed branches of LookupResolver.query$: aggregation stopping at maxOutputs across hosts, the evidenceLimits shorthand defaulting its byte budget, the bounded candidate scan (scan window, malformed entries, maxHosts), per-source quotas for additional hosts, SLAP discovery refusing bytes that would breach maxTotalBytes, byte reports arriving after cancellation, cancellation dropping an in-flight peer answer and skipping queued hosts, idempotent double cancellation, the concurrent-query ceiling, over-budget answers being dropped without blaming the host, the maxTrackers budget, advertisement-map eviction, and a broader host cache surviving a tighter rediscovery. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nches Adds cases for isBoundSteak rejecting a non-object STEAK entry, an entry missing outputsToAdmit, a STEAK record that omits the plan's own decision topic, and an output index that fails to parse while binding STEAK; also covers validateAdmissionPlan rejecting a non-canonical expected-history fence. Two branches in samePayload's optional chaining are confirmed unreachable (they require a payload ref with an undefined digest, which the type system and every real caller already rule out). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… Engine's no-callback path EngineAdmission.test.ts: covers the top-level STEAK pass carrying a caller-supplied coinsRemoved through unchanged for a topic the accepted loop never re-derives, and classifyCoins recovering a previous txid from sourceTransaction.id() when sourceTXID is absent. Engine.admission.test.ts: covers acknowledgeOverlayAdmission's early return when no topic was accepted and no onSteakReady callback was supplied, so the optional-callback branch is exercised without throwing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n via injected runner
New file, mocked MongoTransactionRunner, no real MongoDB: covers
isMongoWriteConflict's non-object/null guard, waitForPending's 50-poll
timeout falling through to a fresh commit attempt, and non-Mongo/non-Error
rejections from the transaction body propagating unclassified rather than
being retried or swallowed. Confirms the write-conflict loop's post-loop
fallback (`return { state: 'rejected', code: 'spend-conflict' }`) is dead
code: the loop's own bound guarantees it always returns/throws by the 8th
attempt, so that line can never execute.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…anches Adds real-replica-set coverage for: a fence conflict rejecting a stale expected history; re-admitting the same tx/topic under a different operation id (fenced off via assertAppliedAvailable, exercising the idempotent-spend return in applySpend along the way); a real eviction succeeding; output/applied-history insert collisions caught as invalid-plan, plus their non-duplicate-key-error fallthroughs (via a namespace-scoped injected write failure); omitting firstSeenHeight; upsertTransaction's raw/manifest presence branches; an unrecognized payload kind rejected while pinning; a payload-store failure while pinning classified as payload-not-ready, and an unrelated one left unclassified; claimOutbox tolerating a since-deleted pinned payload; reconcileAdmission delegating to (and reusing) a scoped peer; applyHandoff's happy path, fence mismatch, and missing-lease paths; and two best-effort concurrent races (generation bootstrap, outbox event id) that each still resolve to exactly one winner and one clean rejection. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Covers: reusing a caller-supplied payload store instead of constructing one; insertOutput recording a spent output from the start; findOutput with includeBEEF leaving beef unset when no transaction row exists, and when the transaction row names a payload that is no longer ready; readScript returning an empty script for a (schema-legacy, simulated via bypassDocumentValidation) output document with no scriptPayloadId; and readBeef hydrating a real merkle path pinned by an admission commit into an atomic BEEF, then falling back to the plain BEEF when that merkle payload is later removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
New file, scoped to read()/readBytes() only per this PR's worklist (a concurrent branch owns the rest of MongoPayloadStore.ts's coverage, in its own MongoPayloadStore.test.ts, so this stays a separate file to keep the eventual merge conflict-free). Covers: read() on missing content; read() rejecting a negative offset, a negative length, and an offset+length past the end; read() rejecting a range whose end would exceed a safe JS integer; readBytes() downloading GridFS-spilled content across chunks (including a partial, mid-file range); and readBytes() failing closed on a missing GridFS file and on a ready record with neither inline data nor a file id. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t pin coverage Two follow-ups to the prior MongoAdmissionStorage commit: the unrecognized-payload-kind test kept the output's script offset/byteLength pointed at the original (25-byte) script, so isInvalidOutput's own range check rejected the plan before pin() was ever reached -- fixed to match the substituted payload's declared byteLength, which now actually reaches and exercises pin()'s isPayloadKind guard. Also adds coverage for replaceHistoryPin tolerating a previous history-update pin whose payload document has since been garbage collected while its reference row remains, skipping the release instead of dereferencing a missing document. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds real jest tests for previously-uncovered MongoPayloadStore branches: pin re-add no-op, claimGarbage/finishGarbage edge cases (never-published digest, afterDeleteClaim hook, ownership-metadata drift), lease-stolen fencing before the ready CAS and before GridFS staging, wrong explicit txid on both the fresh-upload and already-ready paths, GridFS crash-boundary hooks, abandonUpload's guard against deleting a concurrently-published file, digest-mismatch-after-spillover cleanup, stale inline-upload recovery, an expired crashed reservation reclaiming and retiring its orphaned GridFS file, and the abort-signal fallback messages used when a signal's `reason` is unset. Also stresses a many-way concurrent first-time publish of the same digest and a corrupted-in-place staged GridFS chunk to exercise the duplicate-key race and post-upload verification failure paths. Uncovered lines for this file drop from 57 to 21 (lcov, line+branch). Remaining lines are either dead code given existing invariants (the per-chunk maxPayloadBytes check, verifyGridFs's non-byte-chunk guard, the "stream writer missing" guard) or depend on timing-sensitive races with no exposed test seam (recoverStaleUpload's current-state guard, the exact instant a GridFS write signals backpressorem). A test also surfaced a real defect, reported separately: finishGarbage's FileNotFound-swallow regex never matches the mongodb driver's actual "File not found for id X" message, so a benign already-deleted-file race during cleanup always rethrows instead of being swallowed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dges Adds real jest tests for previously-uncovered MongoTransactionRunner branches: many concurrent fresh claims on the same operation id (forces the duplicate-key insert race and the "lost the claim" observer path), a claim-row insert failure whose error code is not a duplicate key (must propagate, not be swallowed), reconcile() with no attemptId against a never-created operation (the "unlocated" fallback), reconcile() with a mismatched attemptId against a genuinely pending claim, and commit failures whose deadline elapses mid-retry-loop (the budget-aborted break, verified via a real blocked commitTransaction). Uncovered lines for this file drop from 18 to 9 (lcov, line+branch). Remaining lines depend on races or invariants not reachable from the public API: Budget's manual deadline check is shadowed by the earlier throwIfAborted() once the matching timer fires; abort()'s "already out of transaction" branch has no path that reaches it before the catch that calls it; the post-insert re-read uses a majority/primary read right after an acknowledged write, so it can't observe a miss without breaking the replica set's own consistency guarantee; and the ownership-fencing mismatches and the "operation row disappeared" branches require deleting a submission-operation row, which nothing in this class ever does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds real jest tests for previously-uncovered MongoSchema branches: concurrent bootstraps of a fresh database (races the NamespaceExists and duplicate-ledger swallow paths in createCollectionIfMissing, ensureGridFs, and ensureLedger), a concurrent re-bootstrap after dropping just the GridFS bucket collections, an existing collection whose validator matches but omits an explicit collation (the collation-compatible-by-omission branch), an index-creation failure whose error code is not one of the benign conflict codes, and a ledger insert failure whose error code is not a duplicate key (both must propagate rather than being swallowed). Uncovered lines for this file drop from 11 to 10 deterministically; the concurrent-bootstrap races additionally land on the NamespaceExists swallow branches in some runs but not every run, since they depend on genuine scheduling of concurrent createCollection calls. The remaining lines are dead code given the surrounding invariants (decodeMongoUint64's re-encode check, the unused `string()` default length and `allowAdditional` option on module-private helpers no call site ever exercises) or require racing a second writer against transactionalProbe's own read-then-CAS window, which was not attempted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> # Conflicts: # packages/sdk/browser-budget.json # packages/sdk/docs/reference/overlay-tools.md # packages/sdk/src/overlay-tools/LookupResolver.ts # packages/wallet/wallet-toolbox/client/platform-budget.json # packages/wallet/wallet-toolbox/mobile/platform-budget.json
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…Garbage finishGarbage is documented as safe to retry, but it only swallowed errors matching /FileNotFound/. The MongoDB driver reports a missing file as MongoRuntimeError "File not found for id <id>", which never matched, so a finisher that lost the race to a concurrent retry rethrew and left the payload row in 'deleting'. Match the driver's actual message. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> # Conflicts: # docs/packages/overlays/overlay-discovery-services.md # docs/packages/overlays/overlay-express.md # docs/packages/overlays/overlay-topics.md # docs/packages/overlays/overlay.md # docs/reference/package-api-migrations.md # docs/reference/stack-facts.md # governance/package-release-notes.json # governance/repository-health/baselines.json # packages/overlays/overlay-discovery-services/CHANGELOG.md # packages/overlays/overlay-discovery-services/package.json # packages/overlays/overlay-express/CHANGELOG.md # packages/overlays/overlay-express/package.json # packages/overlays/overlay/CHANGELOG.md # packages/overlays/overlay/package.json # packages/overlays/topics/CHANGELOG.md # packages/overlays/topics/package.json
BRC-136 defines an empty topic at a height as k = 0 with R = 32 zero
bytes, and rebuildTopicAnchorChain anchors every height that way. The
reconciler accepted such an anchor and then requested a compound Merkle
path for an empty txid list, which BASMRemote rejects ("At least one
BASM proof txid is required"). Any node behind a peer aborted at the
first height the topic admitted nothing, which is most heights.
After the root and count check, an empty admitted list has nothing to
bind or fetch: count the height as checked and skip the proof and raw
transaction round trip. It diverges only when this node admitted
transactions at that height. A zero-count anchor with a non-zero root is
still rejected.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> # Conflicts: # docs/packages/overlays/overlay-discovery-services.md # docs/packages/overlays/overlay-express.md # docs/packages/overlays/overlay-topics.md # docs/packages/overlays/overlay.md # docs/reference/package-api-migrations.md # docs/reference/stack-facts.md # governance/package-release-notes.json # governance/repository-health/baselines.json # packages/overlays/overlay-discovery-services/package.json # packages/overlays/overlay-express/package.json # packages/overlays/overlay/CHANGELOG.md # packages/overlays/overlay/package.json # packages/overlays/topics/package.json
getOverlayAdmissionHost copied publishAdmissionPayload, enlistedIndexTargets and getHistoryFence off the storage as detached function references. Storage adapters are class instances, so inside MongoOverlayStorage.publishAdmissionPayload `this` became the plain host object and `this.payloads.publish` threw a TypeError: the Engine admission path crashed on the first payload with the real Mongo adapter. Every test reaching that path had mocked the method with jest.fn(), which ignores its receiver. Bind each optional method to the storage. Cover it with a class-based unit test and a real replica-set test that publishes through the host object the Engine uses; both failed before the change. The existing helper test asserted function identity, which is the defect, and now asserts that calls are forwarded with the storage as receiver. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overlay reliability program
This change is one slice of Make Overlays Great Again.
Do not merge until the stacked TypeScript and Go overlay branches have been proven together locally. Isolated CI green is not integration acceptance. Keep this PR draft. No npm publication, live database migration, or Mongo/BASM default activation.
C05 (verified progressive API) is parked. C02 (ts-stack#517) and C04 (ts-stack#518) both edit
LookupResolver.tsand must not merge independently.Program and scope
d836611b71fdfbf2dbd521eacc3eb0685109cac8codex/overlay-mongo-s02(PR Add opt-in Mongo overlay payload and schema foundation #519)Impact
Affected packages:
@bsv/overlayVerification
Security and dependencies
Release and operations