feat(pkl): extract calls, branches, throws and decorators - #1426
Open
tthayer wants to merge 2 commits into
Open
Conversation
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
tthayer
force-pushed
the
feat/pkl-call-extraction
branch
2 times, most recently
from
August 3, 2026 18:04
f9cc2b9 to
54a64c8
Compare
Pkl was wired end-to-end but shallow: on a real-world 172-file Pkl corpus
the graph had 386 IMPORTS edges and 0 CALLS edges, because pkl_call_types
was empty_types.
Pkl has no dedicated call node. `unqualifiedAccessExpr` and
`qualifiedAccessExpr` are the same node whether they are a call
(`helper(a)`) or a bare property read (`host`) — the only discriminator is
an `argumentList` child. extract_pkl_callee gates on that child, and is
dispatched from extract_callee_name with an unconditional return: falling
through to field-based or generic first-identifier resolution would mint a
CALLS edge for every property read in every Pkl file, since a bare access
expr's first child is an identifier.
Callee resolution:
helper(a) -> "helper"
utils.fallback(a) -> "utils.fallback" (module-qualified; cbm.c
shortens to the last dotted segment)
s.trim().toLowerCase() -> "toLowerCase" (a receiver that is itself a
call has parens in its text and is not prefixed)
new Server { ... } -> "Server" (links to the class def)
Also fills the other empty slots in the Pkl spec, all confirmed against
real parse trees rather than guessed: typeAlias (class), importGlobClause /
importExpr (import), ifExpr / whenGenerator / forGenerator (branch),
throwExpr (throw), annotation (decorator). forGenerator is additionally
registered in cbm_is_loop_node_type so `for (x in xs)` counts toward
loop-nesting depth; the name is Pkl-unique so no other grammar collides.
The Pkl repro test moves from the structural battery (dims 1-5) to the
full callable battery (dims 1-8) and adds an inline negative assertion
that bare property reads do NOT become CALLS edges — the regression the
argumentList gate exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Tony Thayer-Osborne <tony.thayerosborne@conductorone.com>
tthayer
force-pushed
the
feat/pkl-call-extraction
branch
from
August 3, 2026 18:29
54a64c8 to
4e8d01d
Compare
tthayer
marked this pull request as ready for review
August 3, 2026 19:47
The Windows shard failed Step 0i (parallel suite scheduler contract) with "cleanup failed: hang_after_summary: leader exited leaving live descendants" while its sibling shard passed the same contract. Nothing in the wave was actually leaking a process. run-test-wave.py passed --kill-grace as the subprocess timeout for the two external Windows helpers: taskkill.exe /T /F and the powershell.exe Get-CimInstance descendant probe. Those are different quantities. kill_grace budgets how long a doomed process may take to die; the helpers also have to pay process spawn plus, for PowerShell, CIM startup. The contract fixtures run with --kill-grace 1, and one second is not reliably enough to launch either helper on a loaded runner. The two timeouts then compounded. A timed-out taskkill is reported as "could not prove process-tree cleanup", which raises out of the wave loop; the finally-block cleanup re-enters with the leader already dead, so it falls to the descendant probe, which times out on the same one-second budget and takes its "cannot prove absence -> assume the worst" branch. Phantom descendants, red wave. Give helper invocations their own floor, max(kill_grace, 30). kill_grace still governs every actual death wait, so nothing fails open: the timeout-race contract still refuses with rc=2 over a genuinely surviving descendant, now because PowerShell answered rather than because it timed out. The production path already passed --kill-grace 15 and is unchanged in behaviour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Tony Thayer-Osborne <tony.thayerosborne@conductorone.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Pkl was already wired end-to-end — grammar vendored,
CBM_LANG_PKL, extension mapping, import edges — but the spec was shallow.pkl_call_typeswasempty_types, so on a real-world 172-file Pkl corpus the graph had 0 CALLS edges while imports worked fine.branching_node_types,throw_node_typesanddecorator_node_typeswere empty too, so Pkl contributed nothing to complexity metrics.Why this needs a custom extractor
Pkl has no dedicated call node.
unqualifiedAccessExprandqualifiedAccessExprare the same node whether they are a call or a bare property read — the only discriminator is anargumentListchild:extract_pkl_calleegates on that child. It is dispatched fromextract_callee_namewith an unconditional return, not viaextract_callee_lang_specific: returning NULL from the latter falls through to the generic "first identifier child" fallback, which would mint a CALLS edge for every property read in every Pkl file.Callee resolution:
helper(a)helperutils.fallback(a)utils.fallback(module-qualified;cbm.cshortens to the last dotted segment)s.trim().toLowerCase()toLowerCase(a receiver that is itself a call has parens in its text, so it is not prefixed)new Server { ... }Server(links to the class def)Other spec slots filled
All node names confirmed against real parse trees from a probe linking only the Pkl grammar, not guessed:
+ typeAlias+ importGlobClause,importExprifExpr,whenGenerator,forGeneratorthrowExprannotationforGeneratoris also registered incbm_is_loop_node_typesofor (x in xs)counts toward loop-nesting depth. The name is Pkl-unique, so no other grammar collides.Validation
Repro battery — the Pkl test moves from the structural battery (dims 1-5) to the full callable battery (dims 1-8), plus an inline negative assertion that bare property reads do not become CALLS edges.
repro_grammar_config_pklPASS.Full suite: 318 passed, 6 failed, 2 skipped — byte-identical to the pre-change baseline (verified by stashing and re-running). The 6 failures are pre-existing and unrelated (
repro_issue480/514/546/581,repro_issue409,repro_ts_inherited_method).Real corpus — 172 Pkl files of production infrastructure config, indexed clean (0 skipped, 0 parse failures):
Split: 270 to
Function, 97 toClass(constructors). ~2.1 calls/file — proportionate; a broken gate would have produced tens of thousands.Quality was spot-checked rather than inferred from the count: sampled
Functioncallees are genuinefunctiondefinitions invoked with arguments, and a bare property read that shares its name with one of those functions correctly produced no edge — the exact case theargumentListgate exists to catch.Note for reviewers
The repro fixture deliberately keeps every call site inside a method body, because dim 7 treats Module-sourced in-body calls as the enclosing-func gap. Real-world Pkl does call at module level (many of the 97 constructor edges are such sites), so this is a fixture constraint, not a claim about the language.
🤖 Generated with Claude Code