Skip to content

Bugfix "standalone emoji" by wcwidth grapheme integration - #3956

Closed
jquast wants to merge 2 commits into
Textualize:mainfrom
jquast:jq/wcwidth-integration
Closed

jquast wants to merge 2 commits into
Textualize:mainfrom
jquast:jq/wcwidth-integration

Conversation

@jquast

@jquast jquast commented Feb 2, 2026

Copy link
Copy Markdown

Type of changes

  • Bug fix
  • New feature
  • Documentation / docstrings
  • Tests
  • Other

AI?

  • AI was used to generate this PR

Checklist

  • I've run the latest black with default args on new code.
  • I've updated CHANGELOG.md and CONTRIBUTORS.md where appropriate (see note about typos above).
  • I've added tests for new code.
  • I accept that @willmcgugan may be pedantic in the code review.

Description

Add wcwidth>=0.5.3 as a runtime dependency, and replace Rich's custom code generation and grapheme splitting / width logic measurement with wcwidth.width(), iter_graphemes(), and clip().

Most importantly this fixes some eastern languages with complex combining rules (Virama, Hangul), "standalone" emojis skin tones, standalone "regional indicators", skin tones), but also eliminates ongoing maintenance of custom Unicode logic.

Before

Konsole:
before_konsole

Ghostty:
before_ghostty

After

Konsole:
after_konsole

Ghostty:
after_ghostty

@jquast
jquast force-pushed the jq/wcwidth-integration branch from 7635ba9 to 95586ff Compare February 2, 2026 23:01
@jquast

jquast commented Feb 2, 2026 •

Copy link
Copy Markdown
Author

Regarding multiple unicode versions, I do think support for them should be removed.

Six years ago I wrote about the need for and implemented unicode_version parameter to wcwidth, but initial and ongoing ucs-detect results lead me to suggest that unicode support varies widely[1][2], not just by version, but by feature -- ZWJ/RI/VS16/Graphemes/Languages. Beginning wcwidth 0.5, only single-version tables are published derived from the latest Unicode data release files (17.0)

Although it is possible to introspect each terminal, as the ucs-detect tool does, to test against and curate "prediction tables" to absolutely match all terminals, supporting or not, eg. Supports v12 zero tables, v17 wide, ZWJ but with these exceptions, no VS-16 support, limited grapheme support, the overhead is considered too great to support by code.

The python wcwidth library specifications very closely matches the kitty specification and is maintained with the help of community to match development of actively maintained terminal emulators. The nightly of ghostty matches this specification very closely and has a perfect score with ucs-detect tool, as pictured in the "After" screenshot, above without error.

@jquast

jquast commented Sep 29, 2026 •

Copy link
Copy Markdown
Author

This project appears abandoned, but I would really recommend to take a peek at the latest version of wcwidth should it ever return active

Most recently I improved performance ~50x by use of C Python build extension, and, we now offer "terminal-specific" correction function, wcstwidth described in more detail here, https://www.jeffquast.com/post/perfecting-terminal-character-width-using-correction-tables/ -- I even use this issue's screenshots as a demonstration in the article

@jquast jquast closed this Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant