treesitter: fixtures for the directives an unreadable construct swallows - #95
Merged
rikvanriel merged 1 commit intoSep 29, 2026
Conversation
`include/linux/dev_printk.h` defines 41 function-like macros and the index
holds one of them. The cause is not the header's `#ifdef`s: a macro defined
once per branch is indexed today. It is that
`static inline __printf(3, 4)` -- an attribute macro between the storage
class and the return type -- is not parseable, and the resulting ERROR node
does not stop at the declaration. It runs to the end of the file and
swallows every directive after it, so no preproc_function_def node exists
for the extraction query to match. In that header the construct is on line
71 and the span is lines 1..290.
Nothing is logged when this happens. The file indexes successfully, 16,125
`dev_dbg` call sites have no target, and tree-wide 1,224 function-like
`#define`s across 435 files do not exist in the index. Measured on Linux
50d05c7c76c9 with tree-sitter 0.26.11 and tree-sitter-c 0.24.2, which is
part of the measurement: 0.26.13 drops 27 more on the same tree.
No behaviour change. These are the shapes, each reduced until it is the
smallest source that still reproduces what the tree does, and each test
states what the parse returns rather than what it ought to:
- the seven lines the defect reduces to;
- dev_printk.h's shape, including a body that continues over a line and a
name defined once per branch;
- a commented-out directive inside the swallowed span, which is how a
recovery that blanks what an ERROR covers can invent a definition the
file never declared;
- a `#define` between enum members, which is lost for a different reason
and whose span does not run on, so recovery must not claim it.
Two of these hold the line for whatever recovery lands next: what parses
today is a baseline it may not lose -- a prototype turned dce_hwseq.h from
59 definitions into 35 by blanking a struct whose tokens the extraction
depended on -- and a macro inside a comment is never a definition, because
a count that goes up is not evidence that what was gained was declared.
A fixture for the fifth shape is missing on purpose. The truncation in
include/drm/intel/pciids.h, a block comment immediately before a
continuation backslash, did not reproduce in two hand-written reductions:
a run of three such directives extracts all of them. It needs a reduction
from the file itself, and a fixture that does not reproduce the mechanism
is worse than none.
Test Plan:
cargo test -- 278 tests, five new, all passing: the three losses above and
the two invariants.
cargo test -- --ignored -- three new tests fail with the recovery they gate
absent, one per recoverable shape, which is what makes them a gate rather
than a description.
cargo clippy --all-targets -- -D warnings is clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
include/linux/dev_printk.hdefines 41 function-like macros and the index holds one of them. The cause is not the header's#ifdefs: a macro defined once per branch is indexed today. It is thatstatic inline __printf(3, 4)-- an attribute macro between the storage class and the return type -- is not parseable, and the resulting ERROR node does not stop at the declaration. It runs to the end of the file and swallows every directive after it, so no preproc_function_def node exists for the extraction query to match. In that header the construct is on line 71 and the span is lines 1..290.Nothing is logged when this happens. The file indexes successfully, 16,125
dev_dbgcall sites have no target, and tree-wide 1,224 function-like#defines across 435 files do not exist in the index. Measured on Linux 50d05c7c76c9 with tree-sitter 0.26.11 and tree-sitter-c 0.24.2, which is part of the measurement: 0.26.13 drops 27 more on the same tree.No behaviour change. These are the shapes, each reduced until it is the smallest source that still reproduces what the tree does, and each test states what the parse returns rather than what it ought to:
#definebetween enum members, which is lost for a different reason and whose span does not run on, so recovery must not claim it.Two of these hold the line for whatever recovery lands next: what parses today is a baseline it may not lose -- a prototype turned dce_hwseq.h from 59 definitions into 35 by blanking a struct whose tokens the extraction depended on -- and a macro inside a comment is never a definition, because a count that goes up is not evidence that what was gained was declared.
A fixture for the fifth shape is missing on purpose. The truncation in include/drm/intel/pciids.h, a block comment immediately before a continuation backslash, did not reproduce in two hand-written reductions: a run of three such directives extracts all of them. It needs a reduction from the file itself, and a fixture that does not reproduce the mechanism is worse than none.
Test Plan:
cargo test -- 278 tests, five new, all passing: the three losses above and the two invariants.
cargo test -- --ignored -- three new tests fail with the recovery they gate absent, one per recoverable shape, which is what makes them a gate rather than a description.
cargo clippy --all-targets -- -D warnings is clean.