Skip to content

test(brain): canary 7 — local/Ollama parity, and the seam that makes it possible - #87

Merged
AetherAI3 merged 1 commit into
mainfrom
test/brain-parity
Aug 19, 2026
Merged

test(brain): canary 7 — local/Ollama parity, and the seam that makes it possible#87
AetherAI3 merged 1 commit into
mainfrom
test/brain-parity

Conversation

@AetherAI3

Copy link
Copy Markdown
Owner

The last of the seven release canaries, and the one that stayed unwritable
longest. LocalBrain spawned a Python module with no injectable transport, so
nothing in this repo could drive it: a search for new LocalBrain across test/
returned nothing, because it was not constructible under test.

Adds a BrainSpawner seam. LocalBrain already speaks a newline-delimited JSON
protocol over stdio, so a fake that speaks the same protocol is indistinguishable
from the real child. Default behaviour is unchanged — the option simply defaults
to node's spawn.

What parity means here is deliberately NOT "identical events". Ollama is a
pure-TypeScript loop; the Python path has a far richer vocabulary (stage, skill,
telemetry, checkpoint). Requiring identical output would be a false contract.

It means a host driving either brain sees the same SHAPE: the same tool calls in
the same order, each carrying an id it can reply to, terminated by exactly one
done. A host loop must not need to know which brain it is talking to. The test
normalizes both transcripts to that view and requires them equal.

Four assertions:

both brains present the same shape to a host loop
every tool_call from either brain carries a replyable id
each brain terminates with exactly one done
a brain that dies without a done still ends its stream

The last one guards a hang rather than a wrong value: a host awaiting a terminal
event that never arrives waits forever.

Worth recording, because it nearly produced a false finding: the first run of
that fourth test hung for ten minutes. The cause was the fake, not the product.
LocalBrain ends its stream on the child's close event, and the fake's kill()
closed stdout without emitting close — something a real child always does. A
fake that is unfaithful about its termination signal turns a correct product
into a hanging test. Fixed in the fake; the product path was right.

Gates at this commit:
npm run typecheck exit 0
npm test 1117 pass / 0 fail

All seven canaries now run on every build:

1 denied mutation test/release_canaries.test.ts
2 live-child cancel test/process_tree.test.ts
3 reconnect replay test/release_canaries.test.ts
4 remote freshness test/worktree.test.ts
5 fake-gh ship test/ship_rail.test.ts
6 cap across reconnect test/release_canaries.test.ts
7 brain parity HERE

…it possible

The last of the seven release canaries, and the one that stayed unwritable
longest. LocalBrain spawned a Python module with no injectable transport, so
nothing in this repo could drive it: a search for `new LocalBrain` across test/
returned nothing, because it was not constructible under test.

Adds a BrainSpawner seam. LocalBrain already speaks a newline-delimited JSON
protocol over stdio, so a fake that speaks the same protocol is indistinguishable
from the real child. Default behaviour is unchanged — the option simply defaults
to node's spawn.

What parity means here is deliberately NOT "identical events". Ollama is a
pure-TypeScript loop; the Python path has a far richer vocabulary (stage, skill,
telemetry, checkpoint). Requiring identical output would be a false contract.

It means a host driving either brain sees the same SHAPE: the same tool calls in
the same order, each carrying an id it can reply to, terminated by exactly one
done. A host loop must not need to know which brain it is talking to. The test
normalizes both transcripts to that view and requires them equal.

Four assertions:

  both brains present the same shape to a host loop
  every tool_call from either brain carries a replyable id
  each brain terminates with exactly one done
  a brain that dies without a done still ends its stream

The last one guards a hang rather than a wrong value: a host awaiting a terminal
event that never arrives waits forever.

Worth recording, because it nearly produced a false finding: the first run of
that fourth test hung for ten minutes. The cause was the fake, not the product.
LocalBrain ends its stream on the child's close event, and the fake's kill()
closed stdout without emitting close — something a real child always does. A
fake that is unfaithful about its termination signal turns a correct product
into a hanging test. Fixed in the fake; the product path was right.

Gates at this commit:
  npm run typecheck   exit 0
  npm test            1117 pass / 0 fail

All seven canaries now run on every build:

  1 denied mutation        test/release_canaries.test.ts
  2 live-child cancel      test/process_tree.test.ts
  3 reconnect replay       test/release_canaries.test.ts
  4 remote freshness       test/worktree.test.ts
  5 fake-gh ship           test/ship_rail.test.ts
  6 cap across reconnect   test/release_canaries.test.ts
  7 brain parity           HERE
@AetherAI3
AetherAI3 merged commit c165be0 into main Aug 19, 2026
5 checks passed
@AetherAI3
AetherAI3 deleted the test/brain-parity branch August 19, 2026 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant