treesitter: say which files gave up their swallowed directives - #97
Merged
rikvanriel merged 1 commit intoSep 29, 2026
Merged
rikvanriel merged 1 commit into
rikvanriel merged 1 commit into
Conversation
Reading the macros an unreadable construct swallows is silent, in both
directions. A file that gains 35 of them looks exactly like a file that gains
none, and a file recovery refuses to read looks like a file with nothing to
read -- which is the same silence that let the defect stand: one logging
header held 41 directives and contributed one row, and nothing anywhere said
so.
Say it, once per file, and only where there is something to say.
- A file whose swallowed definitions were read names them by count.
- A file where the blanked parse no longer reads a definition the file's own
parse does is refused whole, and says that it was: whatever else that
parse found is not trustworthy, so the swallowed macros stay missing and
the file is worth a look rather than a shrug.
- A file where every definition the blanked parse found sits where the file
writes a comment or a string says that too. Refusing to invent a macro is
a different fact about a file than having none to recover, and a count of
what was refused rides along on the line for a file that recovered
something as well.
Silence otherwise, because the population that enters recovery is an order of
magnitude larger than the population it helps: on Linux 28a2bc7211da, 1,639
files are parsed a second time and 139 of them gain anything. Logging the
attempt would bury the result.
Counting what was refused needed one fix behind it. The rows a second reading
finds were deduplicated to one per name before the file was asked which of
them are comments, and the longer body wins that contest: a `#define` inside a
comment can therefore take the name of a real swallowed definition below it,
which both loses a macro the file defines and reports the loss as a refusal.
Ask the file first, then deduplicate, and a refused name is exactly a name no
row of the file declares.
Test Plan:
Three tests read what the analyzer says through a thread-local subscriber, so
they hold the log as a contract rather than describing it: the recovered count
appears with the file, a refusal appears with its count, and a file the parser
reads whole says nothing at all.
cargo test -- 294 tests, twenty-one in this module, none ignored. The
phantom-takes-a-name test was run against a build that deduplicates before
asking the file, and fails there.
cargo clippy --all-targets -- -D warnings is clean.
Volume over Linux 28a2bc7211da, reading every `*.c` and `*.h` with
`SEMCODE_DEBUG=info` -- these go to stderr, and the default filter is `error`,
so a plain run still shows none of them: 154 lines. 139 report what they
recovered and their counts sum to 681, the tree-wide gain; 15 report a
refusal, among them the AMD display header whose second reading drops 24
definitions the first one reads, `include/linux/bpf.h` and
`drivers/char/random.c`. Not one file on this tree reports a refused
definition, either alone or beside a gain: those two lines have fixtures and
no instance here.
That the recovery lines are invisible by default is worth its own decision
later. A count of what a whole run recovered and refused belongs in the
indexer's summary rather than behind a debug filter, and that needs the
analyzer to return what it did instead of only saying it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reading the macros an unreadable construct swallows is silent, in both directions. A file that gains 35 of them looks exactly like a file that gains none, and a file recovery refuses to read looks like a file with nothing to read -- which is the same silence that let the defect stand: one logging header held 41 directives and contributed one row, and nothing anywhere said so.
Say it, once per file, and only where there is something to say.
Silence otherwise, because the population that enters recovery is an order of magnitude larger than the population it helps: on Linux 28a2bc7211da, 1,639 files are parsed a second time and 139 of them gain anything. Logging the attempt would bury the result.
Counting what was refused needed one fix behind it. The rows a second reading finds were deduplicated to one per name before the file was asked which of them are comments, and the longer body wins that contest: a
#defineinside a comment can therefore take the name of a real swallowed definition below it, which both loses a macro the file defines and reports the loss as a refusal. Ask the file first, then deduplicate, and a refused name is exactly a name no row of the file declares.Test Plan:
Three tests read what the analyzer says through a thread-local subscriber, so they hold the log as a contract rather than describing it: the recovered count appears with the file, a refusal appears with its count, and a file the parser reads whole says nothing at all.
cargo test -- 294 tests, twenty-one in this module, none ignored. The phantom-takes-a-name test was run against a build that deduplicates before asking the file, and fails there.
cargo clippy --all-targets -- -D warnings is clean.
Volume over Linux 28a2bc7211da, reading every
*.cand*.hwithSEMCODE_DEBUG=info-- these go to stderr, and the default filter iserror, so a plain run still shows none of them: 154 lines. 139 report what they recovered and their counts sum to 681, the tree-wide gain; 15 report a refusal, among them the AMD display header whose second reading drops 24 definitions the first one reads,include/linux/bpf.handdrivers/char/random.c. Not one file on this tree reports a refused definition, either alone or beside a gain: those two lines have fixtures and no instance here.That the recovery lines are invisible by default is worth its own decision later. A count of what a whole run recovered and refused belongs in the indexer's summary rather than behind a debug filter, and that needs the analyzer to return what it did instead of only saying it.