Don't fold source after an f-string with an unmatched literal bracket - #858
Open
gadievron wants to merge 1 commit into
Open
Don't fold source after an f-string with an unmatched literal bracket#858gadievron wants to merge 1 commit into
gadievron wants to merge 1 commit into
Conversation
gadievron
force-pushed
the
fix/simplify-fstring-fold
branch
from
August 16, 2026 17:53
6256c29 to
2045417
Compare
real_code() left f-strings untouched so their expression offsets stay
real, but that also left any literal/format-spec ()[]{} text in the
source, where the _parens implicit-continuation pass miscounted it as a
real unmatched bracket -- blanking every following newline, desyncing
offsets, and making worder mis-read class-method def headers.
find_definition then returned None/the call site and Rename silently
corrupted the file (renamed the call, orphaned the def) for methods
defined after such an f-string. Module-level defs were immune.
Fix: after leaving f-strings in place, blank any ()[]{} char not covered
by an OP token (tokenize; Python 3.12+/PEP 701 only -- documented no-op
earlier); TokenError/SyntaxError falls back to prior behaviour.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gadievron
force-pushed
the
fix/simplify-fstring-fold
branch
from
August 16, 2026 19:18
2045417 to
896c632
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
simplify.real_code()folds all source after an f-string that contains anunmatched literal bracket (e.g.
f"[{x}",f"({n} items", an ANSI escapef"\x1b[K{x}", an escaped bracef"{{", or a format spec likef"{x:>[}").real_codedeliberately leaves f-strings unblanked so their inner-expressionoffsets stay valid. The subsequent implicit-continuation pass
_parensthencounts the f-string's literal brackets too, so
parensgoes positive andnever returns to zero, and every newline to EOF is blanked. Everything below the
f-string collapses onto one logical line, so any offset-based analysis of that
code is wrong.
real_code's only consumer isworder, so the user-visible effect lands onworder's header predicate: for a class-attribute def whosedeffollowsthe f-string,
find_definitionresolves to the wrong location (or the call site)and
Renamesilently renames the usages while leaving thedefuntouched (orvice-versa) — no exception, no diagnostic. Module-level and nested defs are
immune (they don't go through that predicate).
Fix
_blank_fstring_literal_brackets()tokenizes the source (PEP 701, Python 3.12+)and blanks every
()[]{}character that is not covered by anOPtoken —i.e. the literal/format-spec brackets, and bracket characters that are literal
data inside a nested string. Real expression brackets (
f"{[1,2][0]}",f"{x:{width}}") areOPtokens and are left untouched. Blanking islength-preserving (one char → one space), so all offsets are unchanged; it is
required so the literal bracket stops swaying
_parens. On Python < 3.12 anf-string is one opaque
STRINGtoken and can't be split, so the pass is a strictno-op; a
TokenError/SyntaxErrorfallback leaves the source unchanged onincomplete/invalid mid-edit buffers.
Tests are version-gated with
only_for_versions_higher("3.12")and cover eachtrigger class above plus an end-to-end
find_definition+Renameregression, aset of preservation controls (real brackets untouched), a module-level immunity
control, and the invalid-input fallback.
Known limitation
The blanking pass is gated on rope's existing regex-based f-string detection
(
ignored_regions), which recognises single-line and triple-quoted f-strings butnot single-quoted multiline f-strings (
f"[{1 +⏎2}"); those still fold asbefore. Dropping the gate (tokenizing unconditionally) would close the remaining
case — happy to do that here instead if you'd prefer.
Related
Sibling of the still-open #499 (
patched_astsetsregion[0]=Noneon anunmatched bracket in an f-string): same trigger family, but a different code path
(
patched_astdoes not callreal_code). This PR does not fix #499. Samedesync class as the false-triple-quote bug #248 (fixed by #309).
Checklist