Python: a spliced template's context reaches the file it lands in

[knutwannheden]

A Python template's context parsed the template and nothing else, so a recipe splicing a name the target file did not import emitted code that raises NameError when it runs. It parses, it prints, and it reads as correct in a diff: two recipes in a downstream package shipped that way, their docstrings showing an import the code never emitted and their tests asserting the broken output as expected. Both were found by reading.

A template's context is now also what it needs bound where it lands:

_arg = capture('arg')
_tmpl = template(f"subprocess.run({_arg}, shell=True)", context=["import subprocess"])

class Visitor(PythonVisitor[ExecutionContext]):
    def visit_method_invocation(self, method, p):
        match = _pat.match(method, self.cursor)
        return _tmpl.apply(self.cursor, visitor=self, values=match) if match else method

Applied to out = os.popen('ls'), what the file already binds decides what it gets:

the file binds	the splice reads	imports added
nothing	subprocess.run('ls', shell=True)	import subprocess
import subprocess as sp	sp.run('ls', shell=True)	none
from subprocess import run	subprocess.run('ls', shell=True)	import subprocess — a member import does not bind the module
if TYPE_CHECKING: import subprocess	subprocess.run('ls', shell=True)	import subprocess — a conditional import binds nothing at runtime
import subprocess in the enclosing function	subprocess.run('ls', shell=True)	none — the local import binds it
subprocess in an enclosing scope, bound to anything else	—	refused: no import reaches a shadowed name

A context binding counts only where the template's own code reads it, judged per occurrence, so a name the template binds for itself (a comprehension variable, a parameter) is neither renamed nor imported, and context that exists only to type a capture stays out of the file entirely. What does get imported joins the file's existing import of the same module — from subprocess import Popen becomes Popen, run as r — under whichever name the context or the file aliases it to, and a relative context import stays relative, from . import util naming a sibling module rather than the standard library's.

visitor= is what reaches the file's imports; the cursor is where the result lands, and the two differ for a recipe splicing somewhere other than where it stands — the scope questions are asked at the cursor. Omitting the visitor still works for a template whose context binds no module — context=["MyType = int"], or no context at all — and raises otherwise, naming the modules and the call to make.

Why automatic

TypeScript already treats this as the template's job. Template.resolveBindings(visitor) binds each module the context declares and returns the local name it actually bound; apply({bindings}) renames the template's references to those names; and rewrite(...).tryOn(cursor, node, {visitor: this}) does the whole thing once a pattern matches, throwing if it was given neither a visitor nor bindings (templating/rewrite.ts:65). Python has no tryOn — apply() is the splice path — so the wiring and the refusal both land there.

Where it refuses rather than deconflicts

TS's maybeBind picks another name when the one it wants is taken. This has no such machinery, so it raises in the two cases where emitting the reference would read the wrong object: a scope at the splice site binding the name, and a module-scope import binding it to another module — the second would otherwise rebind a name the file already reads. from subprocess import run as _run in the context is the way out of the latter.

Reconciling the two import styles — collapsing a spliced subprocess.run(...) to run(...) where the file already imports the member — is #8791. It belongs to a normalizer that sees the whole file, not to the splice: binding is per member, so a template reading two members of a partly-imported module can be collapsed for neither use or for one.

Tests

tests/python/template/test_template_imports.py, one per decision — the table's five rows, the module-scope import that a taken name refuses, context the template never references, a name the template binds for itself, a splice inside a function whose import still lands at module scope, and the cursor-only call that raises.

The dotted-module row turned up a second bug, in both halves of one predicate: AddImport._is_referenced and PythonAddImportVisitor.isReferenced looked for the last segment of a dotted module, so import os.path was only ever added to a file already using a name path. import a.b.c binds a.