Skip to content

feat: project phase 2 - #27

Open
RegiByte wants to merge 89 commits into
mainfrom
feat/vm-analyzer
Open

feat: project phase 2#27
RegiByte wants to merge 89 commits into
mainfrom
feat/vm-analyzer

Conversation

@RegiByte

@RegiByte RegiByte commented Jun 7, 2026

Copy link
Copy Markdown
Owner

What's included:

  • Analyzer IR (pure analysis, no evaluation)
  • Bytecode compiler (driven by IR)
  • Linker/Loader phase (handles .clj and .js requires)
  • VM loop with full opcode support

To be done:

  • Evaluator-over-ast
  • Drop initial compiler
  • Drop initial evaluator

The goal of this release is stability and feature completeness, no new features were introduced, the full test suite remains green and gained new tests to validate the Analyzer/IR/VM/Compiler and etc... it's a big release, I wanted to make sure it was stable before publishing a new version.

This PR will serve as the central hub until the list of to-do's is exhausted for this release.

RegiByte and others added 30 commits May 31, 2026 20:22
applyCallableAsyncAst deleted — walkInvokeAsync awaits head + args
(lexical content) then applies sync via ctx.applyCallable for every
callable kind; closures never inherit async. walkLetfnAsync added over a
shared installLetfnBindings (letfn BODY awaits; sibling fn bodies stay
sync). Delegation-list audit: binding inits now await (they are lexical
content). The testing macro de-thunked to the JVM inline binding shape —
the thunk moved the async boundary and broke @ inside testing bodies.
Async probes re-recorded from the live walker: the five formerly
inconsistent propagation cases now pin one identical rejection; 9 F8
probes added. Suite 4903/0 (new baseline), differential 288/288.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
VmExecutionMode returns to its literal meaning: VM participation only.
The walker is THE engine, not a mode. bytecodeBody is attached per mode
(walker :fn op gains the VM fn-body seam, gated on the analyzer's own
zero-captures verdict); the apply hub prefers VM per mode with astMethod
as the unconditional base. Spec re-pins: differential + probes run 'off',
VM machinery specs opt into 'function-body', measurement pins the
VM-refusal→walker fallback chain.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…am deleted

The walker has no fallback engine: analysis errors are ALL fatal at the
evaluation boundary (JVM parity — the malformed-try negative moved from
error_handling_test.clj to a walker probe; differential now 287/287).
Deleted: special-forms.ts, dispatch.ts, collections.ts,
async-evaluator.ts, walker/fn-body.ts, apply's form-body paths (missing
astMethod is now an invariant error), evaluateWithContextInner,
evaluateFormsWithContext + ctx.evaluateForms, the 'interpreter'
event path, and the dead form-path halves in binding-setup/arity/
js-interop. recur-check.ts deletion rides with S5 (vm/compiler.ts
imports it). Bench engines retaxonomized: cljam-walker ('off') +
cljam-vm ('function-body'); cljam-interp/cljam-ast retired.

Gates: suite 4902/0 (= 4903 − clj negative − differential twin + probe),
differential 287/287 with 287/287 VM coverage, typecheck, dist bundle
smoke (sync/async/letfn-async), bench sanity (fib 338ms walker / 443ms VM).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…iler specs on the live IR path

Rewired first (production consumer safety): vm/introspection.ts
tryCompileVm -> tryCompileVmFromIr (bytecode-info* probed green before and
after); compiler-test-utils.ts now exports IR-backed compileVm/tryCompileVm/
tryCompileVmFnBody (bare-env analysis keeps free symbols runtime-resolved);
helpers.ts compiles with its session env/ctx; analyzer-var-lexical.spec
cross-checks the analyzer's lexicalCandidates against the IR chunk.

Deleted: vm/compiler.ts (2,216), ir-parity-utils.ts + 6 ir-parity specs
(proof complete: IR ≡ legacy certified since Phase 1), recur-check.ts
(last consumer was compiler.ts).

Live-contract re-pins the rewire surfaced:
- letfn-star: binding calls go through the captured slot (RB-007 intended
  divergence, no FnRecur self-call); duplicate names compile (last wins);
  ir-compiler now REFUSES non-fn letfn* inits with the walker's exact
  error (was a real walker/VM divergence: VM compiled '(letfn* [f 1] f)'
  to 1 while the walker throws) and names letfn templates like the walker.
- loop-recur: variadic fn recur is strict fixed+1 (analyzer arity rule);
  the final arg packages into the rest slot; legacy extras-repackaging
  leniency retired.

Gate: suite 4773/0 (133 files) — NEW BASELINE = 4902 − 130 parity tests
+ 1 re-pin (arithmetic proves no collateral); differential 287/287 with
287/287 VM coverage; typecheck clean; bytecode-info 18/18 before/after.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
runs/20260706-183941: cljam-walker 12.0x / cljam-vm 11.2x SCI geomean —
statistically identical to the pre-deletion profile; FINDINGS retaxonomy
addendum (cljam-interp/cljam-ast retired as engine names; map-assoc
anomaly now the worst board item).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
recur to a variadic fn method takes exactly fixed+1 args
(analyzer-enforced) and the final arg rebinds the rest slot without
gathering — JVM: "there is no gathering of rest args". Walker gathers
once before the apply-hub loop and rebinds e.args 1:1; ir-compiler
emits plain FnRecur for both shapes. FnRecurRest survives for its one
correct emitter: tail self-CALLS (a self-call is a call).

Fixes the canonical (recur (+ acc (first more)) (next more)) idiom.
Suite 4776/0 (133 files), differential 288/288, typecheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ReaderError catchability depends on WHO invoked the reader: parsing the
program fails before evaluation (no try can observe it, JVM parity), but
read-string and edn/read-string parse USER DATA at runtime — JVM throws
a catchable RuntimeException there. asRuntimeReadError (errors.ts)
translates ReaderError/TokenizerError -> EvaluationError with code
reader/malformed at exactly those two native boundaries; the
thrownValueForHandler host-fault contract is untouched, so cljam's own
internal faults still escape rather than masquerade as catchable values.

Also: RB-007 pinned closed — the session-302 combination shape (let +
volatiles + nested letfn + mutual step/drain across lazy-seq thunks)
passes under analyzer capture sets; regression deftest added. The
release-blockers repro harness (session 298) is now committed and runs
clean: "No release blockers detected in this run."

Suite 4780/0 (133 files), differential 290/290, typecheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both READMEs claimed a pure tree-walker with a deleted incremental
closure compiler ("no compilation step and no bytecode"). Reality since
Phase 4: reader -> analyzer (resolved AST) -> AST walker as THE engine,
with the bytecode VM sharing the same front-end under a differential
harness. Also: structural sharing is real now (trie vectors, HAMT maps),
stdlib list includes edn/math/test, and both READMEs gain the Async
section — the lexical-boundary rule with the JS side-by-side, pending
composition (then/catch*/all), and JVM 3-arg deref. Every documented
behavior was probe-verified against the live session first.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant