Skip to content

fix: keep control characters in content from acting as markers - #81

Merged
hownowstephen merged 1 commit into
mainfrom
fix_marker_injection
Sep 23, 2026
Merged

hownowstephen merged 1 commit into
mainfrom
fix_marker_injection

Conversation

@hownowstephen

@hownowstephen hownowstephen commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Conversion marks blockquotes, list indents, horizontal rules and <pre> blocks with the control characters \x00–\x06. The comment on those constants said the parser never emits control characters, but it does: raw ones pass through, and so do entities such as &#1;. Content containing them was read back as markup.

<p>see &#x01;quote&#x02; here</p>          before: "> see quote here"
                                           after:  "see quote here"

<p>x&#6;y</p><pre>code</pre>              before: "xcodey\n\n\x06"
                                           after:  "xy\n\ncode"

Text and attribute values are now read through withoutMarkers: text nodes, span text, <pre> text and getAttr. It does a byte scan first and only rebuilds a string when a marker is present. Markers are ASCII, so the scan cannot match inside a multi-byte rune.

I first tried a separate walk over the tree before conversion, but that cost about 3% and strings.Map on every string cost 8%. Reading through withoutMarkers costs +1.6% geomean when run interleaved against main, and only pathological is outside noise at +2.3%. Allocations are unchanged.

Output is byte-for-byte identical to main across the corpus under every link style.

@hownowstephen
hownowstephen marked this pull request as ready for review September 23, 2026 04:37
@hownowstephen
hownowstephen merged commit fc3754c into main Sep 23, 2026
2 checks passed
@hownowstephen
hownowstephen deleted the fix_marker_injection branch September 23, 2026 04:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant