Skip to content

fix(document-symbols): unbounded MAP/MODULE scan corrupting the outline - #382

Open
geircodes wants to merge 2 commits into
msarson:version-1.0.2from
geircodes:fix/map-module-symbol-endless-lookahead
Open

fix(document-symbols): unbounded MAP/MODULE scan corrupting the outline#382
geircodes wants to merge 2 commits into
msarson:version-1.0.2from
geircodes:fix/map-module-symbol-endless-lookahead

Conversation

@geircodes

@geircodes geircodes commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Two document-symbol (outline) bugs found while auditing a real production .clw file's Document Structure (outline) panel for a WINDOW procedure — one of them significant beyond just that outline, since it silently corrupted symbol classification for the rest of the file once triggered.

  1. ClarionDocumentSymbolProvider.ts — unbounded MAP/MODULE lookahead. When the parser hits a MAP/MODULE structure, it scans forward to find shorthand procedure declarations, accepting an EndStatement as the structure's true end only if no Structure token had been seen since. A MODULE(...) nested inside a MAP is itself a Structure token, so once one appeared, that condition could never become true again — the scan ran unbounded through the rest of the file. Any later identifier immediately followed by ( (e.g. WINDOW attributes like FONT(...), VALUE(...), FROM(...)) got misclassified as a MAP/MODULE procedure and surfaced as a stray top-level outline entry, for every symbol after that point in the file — not just inside the offending WINDOW.

    Fix: bound the lookahead with the structure's own finishesAt, which the tokenizer already computes with correct nesting depth and which the rest of the codebase (e.g. MapProcedureResolver) already relies on — instead of re-deriving the end with the broken heuristic.

  2. DocumentStructure.ts — single-line structure declarations never got a label. handleStructureToken() computes endsOnSameLine for structures that open and close on one line (e.g. fq QUEUE(FILE:Queue) END) and returns early — before reaching the code further down that reads the preceding same-line token and assigns token.label. Multi-line structures worked fine; any single-line QUEUE/GROUP/etc. never got its declared name attached, so the outline showed a bare "QUEUE" node instead of "QUEUE (fq)".

    Fix: hoisted the label-assignment block (including the Language support: labeled LOOP and BREAK/CYCLE with label target #65 LOOP/ACCEPT special case) to run before the endsOnSameLine check, and removed the now-dead duplicate copy that used to sit after the early return.

The detail-text change was split out of this PR per review — it's proposed separately as its own
look-and-feel issue with a ready patch.

Disabled (commented out, not deleted, for a cheap revert if some consumer turns out to need it) the three call sites that set this text.

Checked git history before writing (1) and (2): both bugs were introduced together, from scratch, in the single commit that created DocumentStructure.ts (#280) — not a regression of previously-correct behavior, and no existing helper elsewhere in the codebase does the same job that could have been reused instead.

Test plan

  • New regression test DocumentSymbol.MapModuleLookahead382.test.ts pinning the exact trigger shape — a MAP containing a nested MODULE, followed by a WINDOW: attribute keywords (FONT/AT/USE/FROM) must not appear as outline symbols. Fails against the unbounded scan (verified by reverting just the provider: the old code emits stray FONT, FONT, FROM entries), and a companion case confirms the MAP's own prototype and the nested MODULE's prototype still index.
  • npx tsc -b — clean compile
  • Full test suite: 2385 passing, 0 failing, 4 pending (pre-existing, unrelated), rebased onto origin/version-1.0.2
  • Verified directly against a real production .clw file (tokenizer + provider run outside the test suite): no stray FONT/VALUE/FROM entries, no bare QUEUE nodes (1488 symbols checked)

🤖 Generated with Claude Code

@msarson

msarson commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Reviewed — the core fix is correct: bounding the MAP/MODULE END-detection scan by finishesAt ?? line is the right call, since the old !tokens.slice(...).some(isStructure) guard becomes permanently false once a nested MODULE appears inside a MAP, running the scan to EOF and mis-marking FONT(...)/VALUE(...) as MAP procedures.

Holding for two reasons, both process rather than logic:

  1. Unrelated scope-creep. The change also disables the "in <Parent>" symbol detail. That's independent of the MAP/MODULE scan bug and shouldn't ride along in the same PR — please split it out (or open a separate PR with its own rationale).
  2. No regression test. This is an easily-testable bug (a MAP containing a nested MODULE followed by non-procedure tokens), and our bugfix workflow is red-TDD. Please add a test that fails before the bound and passes after.

Split #1 out and add the test for #2 and I'll merge. 👍

@msarson msarson left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Core fix is correct — holding on two process points: an unrelated "in <Parent>" detail change bundled in, and no regression test. Details in the review comment above.

@github-actions
github-actions Bot deleted the branch msarson:version-1.0.2 August 9, 2026 11:01
@msarson
msarson changed the base branch from version-1.0.1 to version-1.0.2 August 9, 2026 11:32
@msarson

msarson commented Aug 9, 2026

Copy link
Copy Markdown
Owner

👋 Housekeeping: 1.0.1 is now released (live on the Marketplace), so version-1.0.1 has been merged to master and the branch removed. I've retargeted this PR to version-1.0.2 — the new active development branch.

This PR is still held on the review feedback above — no change to that. When you get a chance to revise, please rebase your branch onto version-1.0.2 and push. Thanks for the contribution and your patience! 🙏

…tions

handleStructureToken() computed endsOnSameLine for structures that open and
close on one line (e.g. `fq QUEUE(FILE:Queue) END`) and returned early —
before reaching the code further down that reads the preceding same-line
token and assigns token.label. Multi-line structures worked fine; any
single-line QUEUE/GROUP/etc. never got its declared name attached, so the
outline showed a bare "QUEUE" node instead of "QUEUE (fq)".

Hoisted the label-assignment block (including the msarson#65 LOOP/ACCEPT special
case) to run before the endsOnSameLine check, and removed the now-dead
duplicate copy that used to sit after the early return.
…finishesAt

The lookahead accepted an EndStatement as the structure's true end only if no
Structure token had been seen since - but a MODULE nested inside a MAP is
itself a Structure token, so that condition could never become true again
once one appeared, and the scan ran unbounded through the rest of the file.
Any later identifier immediately followed by '(' (e.g. WINDOW attributes like
FONT(...), VALUE(...), FROM(...)) got misclassified as a MAP/MODULE procedure
and surfaced as a stray outline entry.

Bound the lookahead with the structure's own finishesAt, which the tokenizer
already computes with correct nesting depth (and which the rest of the
codebase, e.g. MapProcedureResolver, already relies on) - instead of
re-deriving the end with the broken heuristic.

The 'in <Parent>' detail suppression that previously rode along in this
branch is removed - it is independent of this fix and will be proposed
separately.

Tests: new DocumentSymbol.MapModuleLookahead382.test.ts pins the exact
trigger shape - a MAP with a nested MODULE followed by a WINDOW: attribute
keywords must not appear as outline symbols (fails against the unbounded
scan with stray FONT/FONT/FROM entries), and the MAP's and nested MODULE's
real prototypes must still index. Full suite 2385 passing.
@geircodes
geircodes force-pushed the fix/map-module-symbol-endless-lookahead branch from d493f5a to 1959f62 Compare August 14, 2026 06:52
@geircodes

Copy link
Copy Markdown
Contributor Author

Reply for PR #382 (fix/map-module-symbol-endless-lookahead)

Both points addressed; rebased onto version-1.0.2.

1. Scope creep removed. The "in <Parent>" detail suppression is out of this PR — the branch now carries only the outline-correctness fixes (the finishesAt bound, plus the single-line-structure label attach from the first commit). I'll propose the detail-text change separately with its own rationale, as a look-and-feel decision for you to make.

2. Regression test added — new DocumentSymbol.MapModuleLookahead382.test.ts pinning the exact trigger shape (a MAP containing a nested MODULE, followed by a WINDOW with attribute keywords):

  • attribute keywords must not appear as outline symbols — fails against the unbounded scan (verified by reverting just the provider: the old code emits stray FONT, FONT, FROM entries, exactly the reported corruption);
  • the MAP's own prototype and the nested MODULE's prototype must still index (guards against over-bounding).

Full suite: 2385 passing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants