Skip to content

feat(extract): add a Twig template extractor#2001

Open
nitrobear wants to merge 1 commit into
Graphify-Labs:v8from
nitrobear:feat/twig-extractor
Open

feat(extract): add a Twig template extractor#2001
nitrobear wants to merge 1 commit into
Graphify-Labs:v8from
nitrobear:feat/twig-extractor

Conversation

@nitrobear

Copy link
Copy Markdown

Twig carries the entire presentation layer of Symfony, Drupal, Craft CMS and Grav projects, and none of it reached the graph: .twig was absent from CODE_EXTENSIONS, so the files were never walked in the first place. On a mid-sized Symfony app that is ~143 templates — the whole admin UI — invisible to query/explain/path.

The extractor follows the existing regex-based template extractors (blade.py, razor.py) rather than adding a grammar:

  • inheritance and composition: {% extends %}, {% include %}, {% embed %}, {% import %}, {% from %}, {% use %}, plus the {{ include(...) }} function form
  • {% block name %} definitions, scoped per file so that two templates defining content do not collapse onto one node
  • path() / url() calls as references_route edges

That last relation is the one that pays: a route name emitted by a template resolves to the controller declaring it, so the presentation layer stops being a disconnected island in the graph.

Two details worth flagging for review:

  • References resolve against the nearest ancestor directory named templates, which is the loader root convention in all four frameworks. When the target file exists, the node reuses that file's id, so {% extends "base.html.twig" %} becomes a real template-to-template edge instead of a dangling label. Namespaced (@AcmeBundle/...), out-of-tree and dynamic names stay as standalone label nodes.
  • include() / path() / url() are only scanned inside {{ ... }} and {% ... %} regions. Templates carry a lot of inline <script>, and an unscoped scan happily matched a JS function literally named path(.

Covered by tests/test_twig_extraction.py (15 tests): dispatch, extension registration, each relation, whitespace-control markers ({%-), per-file block scoping, line numbers, the inline-script false positive above, unresolvable references, and unreadable files.

Twig carries the entire presentation layer of Symfony, Drupal, Craft CMS and
Grav projects, and none of it reached the graph: `.twig` was absent from
CODE_EXTENSIONS, so the files were never walked in the first place. On a
mid-sized Symfony app that is ~143 templates — the whole admin UI — invisible
to `query`/`explain`/`path`.

The extractor follows the existing regex-based template extractors
(blade.py, razor.py) rather than adding a grammar:

- inheritance and composition: `{% extends %}`, `{% include %}`, `{% embed %}`,
  `{% import %}`, `{% from %}`, `{% use %}`, plus the `{{ include(...) }}`
  function form
- `{% block name %}` definitions, scoped per file so that two templates
  defining `content` do not collapse onto one node
- `path()` / `url()` calls as `references_route` edges

That last relation is the one that pays: a route name emitted by a template
resolves to the controller declaring it, so the presentation layer stops being
a disconnected island in the graph.

Two details worth flagging for review:

- References resolve against the nearest ancestor directory named `templates`,
  which is the loader root convention in all four frameworks. When the target
  file exists, the node reuses that file's id, so `{% extends "base.html.twig" %}`
  becomes a real template-to-template edge instead of a dangling label.
  Namespaced (`@AcmeBundle/...`), out-of-tree and dynamic names stay as
  standalone label nodes.
- `include()` / `path()` / `url()` are only scanned inside `{{ ... }}` and
  `{% ... %}` regions. Templates carry a lot of inline <script>, and an
  unscoped scan happily matched a JS function literally named `path(`.

Covered by tests/test_twig_extraction.py (15 tests): dispatch, extension
registration, each relation, whitespace-control markers (`{%-`), per-file block
scoping, line numbers, the inline-script false positive above, unresolvable
references, and unreadable files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant