Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,13 @@ Use `knowledgeReleaseReport()` before promotion. It folds the candidate and base
- Research state is durable state. A driver that accumulates belief across rounds takes a store and a `ledgerId` (`createPersistentResearchDrivingDriver`) so corroboration counts, contradiction edges, and open questions survive the process. `runVerifiedResearchLoop` durably announces a fold before its synchronous question generation, calls `driver.checkpoint()` before publishing the round event, and reconstructs an interrupted fold on resume.
- `TrackedClaim` remains the live Set-based driver API; `ResearchClaimRecord` is its sorted-array durable form. Convert at the persistence boundary rather than changing the published live shape.
- More than one writer per ledger means `mergeClaimLedger(id, merge)`, never `putClaimLedger`. `putClaimLedger` writes the whole record, so two writers accumulating into one ledger each write what they built from a stale read and the later write erases the earlier writer's claims. `mergeClaimLedger` holds the store's lock across read, merge, and write; `mergeClaimLedgers` is the combining rule and is commutative, associative, and idempotent, so replay and arrival order cannot change the result.
- One lock per store root, and a consumer with its own lock wrapper joins it rather than building a second one.
`withKnowledgeMutation` is reentrant per async context, `isKnowledgeMutationHeld(root)` reports whether this context already holds the root, and `runInKnowledgeMutationScope(root, hold, body)` enters the scope on a lock the caller took by its own path.
Inside either, every lock-taking function in this package runs inline instead of blocking against a lock the caller already holds.
A second lock over the same root is a second writer, whatever lockfile it uses.
- Grade a claim's re-executed check with `gradeFor`, and a whole pass with `gradeClaims`.
The verdict lattice is calibrated: a check that carries its own expected value, one killed at its deadline, and one whose output says it never reached its input are all refusals or environment verdicts, never a refutation of the claim.
`gradeClaims` adds the one judgment a single claim cannot make — it flags a later claim that repeats an earlier claim's check, expectation and title at the same verdict, so one verification counted N times is visible rather than silent.
- Use `writeFileDurable` / `writeJsonDurableWithinRoot` from the entrypoint for any file that must survive a crash. They are atomic, fsynced, and symlink-safe; a hand-rolled `writeFile` is none of those.
- Use `KnowledgeDiscoveryDispatcher` for research workers. Applications should connect it to their own runtime.
- Do not bypass `lint` or `validate` before using generated knowledge in an agent.
Expand Down
50 changes: 50 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,55 @@
# Changelog

## 12.0.0 — 2026-09-01

### Fixed

- A check whose output says it never reached its input is graded `unrunnable` whatever its exit status.
The signature list was consulted only after a nonzero exit.
A shell pipeline exits with the status of its last stage, so `sha256sum <missing path> | cut -d' ' -f1` prints "No such file or directory" and exits 0, and `gradeFor` then compared that error text to the expectation and returned `contradicted` — a refutation of research that was never run.
Swept over 272 grade files, 29 of 61 recorded refutations were this shape: 7 a malformed check body, 7 a shell that could not parse the check, 6 a solver that could not open its problem file, 5 `Permission denied`, 4 `cannot create directory`.
The list now also matches case-insensitively, because a tool that prints its own diagnostic chooses its own case, and it covers `Permission denied`, `EACCES`, `cannot create directory`, `cannot open`, `SyntaxError` and the two bash parse errors.
- The list only ever downgrades `contradicted` to `unrunnable`, so it can turn a measurement that did not happen into an honest "not measured" and can never turn a real failure into a pass.
It consults the expectation first: a claim that predicted the error is genuinely contradicted when the error does not appear, so an expectation naming a signature switches the guard off for that claim.
- A check killed at its deadline is graded `unrunnable`, not `contradicted`.
A deadline is a budget, not a verdict, and the rule fires before the expectation is compared, so a slow solver that printed nothing is no longer recorded as refuted.
- A passing check is no longer refused for naming several values at once.
`expectationRefusalNote` returned `uncheckable` when the expectation carried three or more `key=value` tokens, after the check had exited 0 and immediately above the comparison that would have passed.
The rule inverted its own intent: `expect: OK` is one token and is satisfied by any output containing those two letters, while `GRID OK cells=8 WIN=1 PARETO=6` needs four independent numbers to coincide before it can pass falsely.
Measured, 173 of 1,001 claims carrying an expectation at rung 4 or above — 17.3% — were discarded this way after their check exited 0, including one whose output matched its expectation byte for byte.
Multiline expectations are still refused, because the comparison is one contiguous substring.

### Added

- `gradeFor` and `assertGradeableEvidence` refuse a check whose text contains its own expected value.
Such a check prints a value the claim's subject never had to produce, so the comparison tests the author's typing rather than the artifact.
The test is the whole trimmed expectation appearing verbatim in the command, which is narrow on purpose: it costs an author a rewrite of the command, and it catches the shape `isConstantEmitter` cannot, where a real command's arguments already spell the answer.
- `gradeClaims(claims)` grades one pass and flags the repeats inside it, with `claimCheckKey` for a grader that keeps its own map.
`gradeFor` judges one claim and cannot see a second claim carrying the same check, so a run that recorded one verification under several titles reported as several independent verifications.
A later claim sharing check, expectation, normalized title and verdict now carries `duplicateOf` naming the first.
The verdict and the count do not change: the flag is for a reader deciding how much evidence a run really produced.
Only a full match is a duplicate, so one check shared by claims that say different things is one instrument used several times.
- `DEADLINE_EXIT_CODE` and `CheckExecution.timedOut`, the two ways an executor reports a deadline kill.
A grader that knows it killed the process sets `timedOut`; a grader holding only an exit status reports 124, which is `timeout(1)`'s status.
`verifyGradeableEvidence` now sets `timedOut` when its own budget killed the check, so the deadline rule fires on this package's own execution path rather than only on a caller's.
A killed process reports no exit status of its own, so it read as 127 and was compared to the expectation like any other failure.
- `isKnowledgeMutationHeld(root)` and `runInKnowledgeMutationScope(root, hold, body)`, with the `KnowledgeMutationHold` type.
`withKnowledgeMutation` is reentrant per async context, but the `AsyncLocalStorage` behind that was module-private, so a consumer holding the store lock through its own wrapper could not enter or observe the scope and calling any lock-taking function from this package inside its wrapper self-blocked against a lock it already held.
The scope is now joinable: inside `runInKnowledgeMutationScope` every lock-taking function in this package sees the root as held and runs inline.
The caller owns acquiring and releasing its lock and supplies the hold, which is asked `assertOwned()` at the same points a lock this package acquired is asked, so an externally held lock carries the same guarantee and not a weaker one.
- `KnowledgeMutationOptions.staleMs` documents what the window costs.
The holder heartbeats at a third of it, so the window also bounds how long a crashed holder wedges every other writer.
The 15-minute default is sized for a knowledge-improvement run that holds the lock across agent turns; measured holds are 9 ms for one page, 586 ms for a 50-page batch, and about 390 ms for the longest promotion in the largest store.

### Changed

- **This is a major bump because two exported shapes moved.**
`CheckExecution` gains optional `timedOut` and `ClaimGrade` gains optional `duplicateOf`.
Both are optional, so a consumer that constructs neither field compiles unchanged, and a consumer that switches exhaustively over `ClaimGrade`'s fields does not.
- **A claim whose check contains its expected value is now refused where it was recorded before.**
`assertGradeableEvidence` throws `UncheckableClaimError` for it, and `gradeFor` returns `uncheckable`.
Record time and grade time read one detector and one set of notes, so a shape the recorder admits is never one the grader refuses.

## 11.0.0 — 2026-08-31

### Changed
Expand Down
12 changes: 10 additions & 2 deletions api-surface.json
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,10 @@
"BuildRetrievalEvalDispatchOptions": "value 2a1fe2b2da73",
"CHECKABLE_RUNG_THRESHOLD": "value 534bcde62c80",
"CITES_INVALIDATED_FIELD": "value 3fff28ee6d1f",
"CheckExecution": "value 079d2aa6b6cd",
"CheckExecution": "value f8a48e381773",
"ChunkingOptions": "value 00fb66d7d155",
"ClaimEvidence": "value f780da49a3ef",
"ClaimGrade": "value c38c514e14fc",
"ClaimGrade": "value a9e6edbc7877",
"ClaimGroundingDriverOptions": "value 4c5ca74e7255",
"ClaimLedgerGoalConflictError": "value c96a14249e23",
"ClaimLedgerMigrationRequiredError": "value 2dc627c3fa60",
Expand All @@ -98,6 +98,7 @@
"CreateKnowledgeToolsOptions": "value 8cf079704a21",
"CreateKnowledgeUseReceiptInput": "value eb120a93f4c7",
"D1Adapter": "value fd8669db1f2e",
"DEADLINE_EXIT_CODE": "value 16590df13e07",
"DEFAULT_KNOWLEDGE_BRIEF_LIMIT": "value a16e63bcd9c7",
"DEFAULT_MEMORY_CLEANUP_TIMEOUT_MS": "value 10d1ef3e7731",
"DEFAULT_PAGES_DIRECTORY": "value 5aa47f8e3dde",
Expand Down Expand Up @@ -133,6 +134,8 @@
"FreshnessMark": "value 93bc4803d9b3",
"FreshnessRecord": "value 65bdf72051de",
"FreshnessTtl": "value e3c26a4f6c3e",
"GradeableClaim": "value bf465a1848d5",
"GradedClaim": "value acfba8ff752d",
"GraphitiMcpClientLike": "value a6c48bcfcdd6",
"GraphitiMemoryAdapterOptions": "value 6226003a629c",
"GraphitiToolNames": "value 65e079641c44",
Expand Down Expand Up @@ -258,6 +261,7 @@
"KnowledgeMemoryBenchmarkTaskKind": "type 33dd3b696a1d",
"KnowledgeMemoryEvent": "type b5b297e3f9b1",
"KnowledgeMemoryFactMatcher": "type 095c9075c340",
"KnowledgeMutationHold": "type 353c820b01ff",
"KnowledgeMutationLock": "type 974f26ecfc42",
"KnowledgeMutationOptions": "type 1ab92dd5d3de",
"KnowledgePage": "value 46af63a0ac13",
Expand Down Expand Up @@ -553,6 +557,7 @@
"chunkMarkdown": "value 3f930d4ba32f",
"citedClaimKey": "value 810f529bf94c",
"citedClaimOf": "value 1b37f8f9af86",
"claimCheckKey": "value f7b71c731a17",
"claimEvidenceId": "value 10ea894124db",
"claimId": "value 4c9eca896a37",
"claimSourceHost": "value a8a18cde9fcc",
Expand Down Expand Up @@ -612,6 +617,7 @@
"formatKnowledgeCitationReference": "value 8c9301778549",
"formatKnowledgeInvalidationProposal": "value 0facdbad3e8e",
"fromAgentCandidateKnowledgeRef": "value 247b544b764e",
"gradeClaims": "value 5a3c1c162258",
"gradeCompanyAgainstText": "value 7470a3d96d31",
"gradeFactAgainstText": "value 9818c38e4ce2",
"gradeFor": "value 79db4a5137a3",
Expand All @@ -628,6 +634,7 @@
"investmentThesisSet": "value 373728f5643d",
"isKernelAnchoredPath": "value 2819fa14db65",
"isKnowledgeMemoryBenchmarkCase": "value 8634e2c95fec",
"isKnowledgeMutationHeld": "value e8f0de09e802",
"isKnowledgePagePath": "value 2819fa14db65",
"isMissingFile": "value 73ca49f2b72a",
"isReachable": "value 980b56e89641",
Expand Down Expand Up @@ -718,6 +725,7 @@
"runAgentMemoryLearningExperiment": "value f11e54703bb5",
"runBoundedMemoryLifecycle": "value 75f3e3b274a0",
"runDiscoveryLoop": "value addb8a464a58",
"runInKnowledgeMutationScope": "value 83a5698000d7",
"runInvestmentThesisTask": "value 2eac673e45ab",
"runKnowledgeBenchmarkSuite": "value ce72ffe18eef",
"runKnowledgeResearchLoop": "value 4d00a0a2e19d",
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@tangle-network/agent-knowledge",
"version": "11.0.0",
"version": "12.0.0",
"description": "Build, search, evaluate, and improve source-backed knowledge bases.",
"homepage": "https://github.com/tangle-network/agent-knowledge#readme",
"repository": {
Expand Down
10 changes: 6 additions & 4 deletions src/claim-evidence-intake.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,16 @@ describe('calibrated static intake refusals', () => {
)
})

it('refuses a brittle multi-value expectation', () => {
expect(() =>
it('records an expectation that names several values at once', () => {
// The strongest expectation available: a false pass needs three independent values to
// coincide, where a one-token expectation is met by any output containing that token.
expect(
assertGradeableEvidence({
rung: 4,
check: 'python3 check.py',
expect: 'rank=12 size=40 verified=true',
}),
).toThrow(/one decisive value per claim/)
}).expect,
).toBe('rank=12 size=40 verified=true')
})

it('refuses the escaped-newline long-first-word shape', () => {
Expand Down
Loading