Skip to content

Don't fold source after an f-string with an unmatched literal bracket - #858

Open
gadievron wants to merge 1 commit into
python-rope:masterfrom
gadievron:fix/simplify-fstring-fold
Open

Don't fold source after an f-string with an unmatched literal bracket#858
gadievron wants to merge 1 commit into
python-rope:masterfrom
gadievron:fix/simplify-fstring-fold

Conversation

@gadievron

Copy link
Copy Markdown

Description

simplify.real_code() folds all source after an f-string that contains an
unmatched literal bracket (e.g. f"[{x}", f"({n} items", an ANSI escape
f"\x1b[K{x}", an escaped brace f"{{", or a format spec like f"{x:>[}").

real_code deliberately leaves f-strings unblanked so their inner-expression
offsets stay valid. The subsequent implicit-continuation pass _parens then
counts the f-string's literal brackets too, so parens goes positive and
never returns to zero, and every newline to EOF is blanked. Everything below the
f-string collapses onto one logical line, so any offset-based analysis of that
code is wrong.

real_code's only consumer is worder, so the user-visible effect lands on
worder's header predicate: for a class-attribute def whose def follows
the f-string, find_definition resolves to the wrong location (or the call site)
and Rename silently renames the usages while leaving the def untouched (or
vice-versa) — no exception, no diagnostic. Module-level and nested defs are
immune (they don't go through that predicate).

>>> from rope.base import simplify
>>> simplify.real_code('f"[{x}"\ndef g():\n    return 1\n')
'f"[{x}" def g():     return 1 '   # 3 newlines collapsed to spaces

Fix

_blank_fstring_literal_brackets() tokenizes the source (PEP 701, Python 3.12+)
and blanks every ()[]{} character that is not covered by an OP token —
i.e. the literal/format-spec brackets, and bracket characters that are literal
data inside a nested string. Real expression brackets (f"{[1,2][0]}",
f"{x:{width}}") are OP tokens and are left untouched. Blanking is
length-preserving (one char → one space), so all offsets are unchanged; it is
required so the literal bracket stops swaying _parens. On Python < 3.12 an
f-string is one opaque STRING token and can't be split, so the pass is a strict
no-op; a TokenError/SyntaxError fallback leaves the source unchanged on
incomplete/invalid mid-edit buffers.

Tests are version-gated with only_for_versions_higher("3.12") and cover each
trigger class above plus an end-to-end find_definition + Rename regression, a
set of preservation controls (real brackets untouched), a module-level immunity
control, and the invalid-input fallback.

Known limitation

The blanking pass is gated on rope's existing regex-based f-string detection
(ignored_regions), which recognises single-line and triple-quoted f-strings but
not single-quoted multiline f-strings (f"[{1 +2}"); those still fold as
before. Dropping the gate (tokenizing unconditionally) would close the remaining
case — happy to do that here instead if you'd prefer.

Related

Sibling of the still-open #499 (patched_ast sets region[0]=None on an
unmatched bracket in an f-string): same trigger family, but a different code path
(patched_ast does not call real_code). This PR does not fix #499. Same
desync class as the false-triple-quote bug #248 (fixed by #309).


Checklist

  • I have added tests that prove my fix is effective
  • I have updated CHANGELOG.md

real_code() left f-strings untouched so their expression offsets stay
real, but that also left any literal/format-spec ()[]{} text in the
source, where the _parens implicit-continuation pass miscounted it as a
real unmatched bracket -- blanking every following newline, desyncing
offsets, and making worder mis-read class-method def headers.
find_definition then returned None/the call site and Rename silently
corrupted the file (renamed the call, orphaned the def) for methods
defined after such an f-string. Module-level defs were immune.

Fix: after leaving f-strings in place, blank any ()[]{} char not covered
by an OP token (tokenize; Python 3.12+/PEP 701 only -- documented no-op
earlier); TokenError/SyntaxError falls back to prior behaviour.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@gadievron
gadievron force-pushed the fix/simplify-fstring-fold branch from 2045417 to 896c632 Compare August 16, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

patched_ast will set region[0] to None when unmatched parenthesis inside string

1 participant