Skip to content

test(tangle): assert every Sandbox limit fits the contract bound it feeds - #315

Merged
drewstone merged 1 commit into
mainfrom
test/producer-consumer-bound-seam
Sep 11, 2026
Merged

test(tangle): assert every Sandbox limit fits the contract bound it feeds#315
drewstone merged 1 commit into
mainfrom
test/producer-consumer-bound-seam

Conversation

@drewstone

@drewstone drewstone commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

The seam test. Three bound mismatches shipped through this package in one week (#311, #312, #314) and none was catchable by a unit test, because each side was individually correct — the SDK writes a legal value, the provider rejects a value it considers illegal, and only the pair is wrong.

What it asserts

For every MAX_* limit the Sandbox SDK ships, read out of the shipped dist (a renamed constant fails loudly rather than silently un-checking), the contract bound it feeds must be at least as large — unless the boundary is recorded as truncating rather than throwing, with the reason. A second case enumerates every shipped limit and refuses an unpaired one.

What it found on first run

  • Three unpaired limits (MAX_WORKSPACE_IMAGE_RESPONSE_BYTES, MAX_WORKSPACE_IMAGE_EXPOSED_PORTS, MAX_TIMER_MS) — traced and classified as SDK-internal guards that never reach a contract schema.
  • Four contract constants were not public. agent-interface exported only CONTRACT_MAX_JSON_BYTES; a package that writes values could not assert its limits fit. All eight are exported now.

Suites

agent-provider-tangle 271 pass. agent-interface 547 of 548 — src/certified-context.test.ts fails identically on unmodified main, measured both ways.

agent-interface 2.7.0 → 2.8.0 (new exports), agent-provider-tangle 1.1.10 → 1.1.11.

🤖 Generated with Claude Code

@drewstone @claude
test(tangle): assert every Sandbox limit fits the contract bound it f…
6d75aae 
…eeds

Every bound defect this package shipped this week lived at the seam between
what the Sandbox SDK writes and what this provider accepts, and none was
caught by a unit test, because each side is individually correct. #311: tool
output written at 4 MiB, validated at 16 KiB. #312: widened to 1 MiB, still
4x and the same failure mode. #314: a turn prompt held to the metadata bound.
143 of 199 children in one Lab archive.

This test asserts the pair, not either side. It reads each producer limit out
of the SHIPPED sandbox dist — a renamed constant fails loudly rather than
silently dropping the check — and asserts it fits the contract bound it feeds,
unless the boundary is recorded as truncating instead of throwing, with the
reason. A second case enumerates every MAX_ limit the SDK ships and refuses an
unpaired one, so a new producer bound cannot arrive undecided.

On its first run it found three limits nobody had paired, and it could not read
four contract constants at all: agent-interface exported only
CONTRACT_MAX_JSON_BYTES. The full set is public now, which is what lets a
package that writes values assert its own limits fit.

agent-provider-tangle suite 271 pass. agent-interface 547 of 548; the one
failure, src/certified-context.test.ts, reproduces identically on unmodified
main.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@drewstone

drewstone commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Verified locally, serialized (--fileParallelism=false), same machine, before opening: agent-provider-tangle 271/271; agent-interface 547/548 with the one failure (src/certified-context.test.ts) reproducing identically on unmodified main. Both packages typecheck clean. Merging on that evidence.

@drewstone
drewstone merged commit c07c9f1 into main Sep 11, 2026
2 checks passed
@drewstone
drewstone deleted the test/producer-consumer-bound-seam branch September 11, 2026 19:29
@drewstone drewstone mentioned this pull request Sep 11, 2026
Merged
drewstone added a commit that referenced this pull request Sep 11, 2026
@drewstone @claude
fix(providers): require an interface that bounds a prompt as content (#…
ba7a885 
…320)

`@tangle-network/agent-provider-tangle` is the code that VALIDATES a turn, so the
agent-interface copy IT resolves decides what a turn may contain — not the copy the
consumer installed at its own top level. It declared `^2.6.1`, which still admits the
release whose bug it was shipped to escape.

agent-interface 2.7.0 (#314) moved every `prompt` field off the 16 KiB metadata bound
(`CONTRACT_MAX_STRING_LENGTH`) onto the 1 MiB content bound, and 2.8.0 (#315) made the
contract bounds public. Measured 2026-09-11: a consumer whose lockfile already held 2.6.1
for this package, and which then raised its own top-level agent-interface to 2.8.0, ended
up with two copies — agent-runtime on 2.8.0, the provider on 2.6.1. Because the provider
validates, children kept dying on the 16 KiB bound with a ZodError at `path: ["prompt"]`
on a stack whose package.json said the bug was fixed: 14 of 34 children of one run and 8
of 52 of another, every one at iterations 0, surfaced as "retained provider execution
requires reconciliation before replacement".

Ranges raised, each decided on whether the package's own validation depends on the change:

  agent-provider-tangle      ^2.6.1 -> ^2.8.0  parses every turn with AgentTurnInputSchema
                                               and every interactive command with
                                               AgentInteractiveSessionPromptCommandSchema;
                                               also imports the bounds 2.8.0 made public.
  agent-provider-cli-bridge  ^2.4.0 -> ^2.7.0  prepareCliBridgeRun parses every turn with
                                               AgentTurnInputSchema. Same exposure, and
                                               2.7.0 is where the bound it needs arrived.

Left alone, each checked rather than assumed:

  agent-core           ^2.0.0  interaction schemas and stream event types only; the
                               interaction-answer bound did not move in 2.7.0.
  agent-provider-{e2b,daytona} ^2.4.0  type-only imports; validate no turn.
  agent-provider-computesdk    ^2.4.0  parses WorkspaceRequestSchema only.
  agent-provider-testkit       ^2.4.0  builds short conformance prompts and asserts digest
                               replay, not size; a prompt digest is content-addressed and
                               identical across the bound change.
  agent-trace-contract         no agent-interface dependency.

`producer-consumer-bounds.test.ts` (#315) cannot see this split: it reads the shipped
Sandbox dist and compares it against contract constants it imports, and the workspace pins
`overrides["@tangle-network/agent-interface"]: workspace:*`, so in this repo it always
links the workspace copy. `resolved-interface-floor.test.ts` checks the other half —
behaviour first, then the versions:

  - `promptFromTurnInput` accepts a prompt one character over the metadata bound;
  - `AgentInteractiveSessionPromptCommandSchema` reports no issue at `path: ["prompt"]`
    for an oversized prompt;
  - the resolved copy exports the contract bounds `producer-consumer-bounds.test.ts`
    imports by name — on an older copy that file does not fail, it fails to LOAD;
  - the agent-interface this package resolves is at or above its declared floor;
  - the declared floor is at or above the release the validation needs, which is what the
    workspace override would otherwise hide.

It imports agent-interface as a namespace on purpose: a named import of a constant an old
resolution does not export fails at link time with a SyntaxError naming no package, no
version and no reason, and this file exists to diagnose exactly that resolution.

Proven against a deliberately old resolution rather than only asserted. The published
provider 1.2.0 installed beside agent-interface 2.6.1, with no override, shares the 2.6.1
copy and refuses a 20,000-character prompt with `too_big maximum 16384 path ["prompt"]`;
this branch's tarball, in the same tree, makes npm nest 2.9.0 under the provider and the
same prompt is accepted unchanged. The new test file, run against an installed 2.6.1,
fails 4 of its 5 cases with those messages, and passes 5 of 5 against the published 2.9.0.

Tests, serialized (`--fileParallelism=false`): agent-provider-tangle 285 pass in 20 files,
up from 280 in 19; cli-bridge 214; testkit 27; agent-core 506; trace-contract 182;
computesdk 6; daytona 4; e2b 4. agent-interface 554 of 555 — `certified-context.test.ts`
fails 1 of 12 identically on unmodified origin/main in this worktree. `pnpm -r check-types`,
`pnpm -r build`, `pnpm check:package-artifacts` (9 packages, 31 exports, 62 export targets)
and `pnpm check:control-artifacts` (183 tests) all pass.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone