libwcwidth, a portable C11 library - #247
Merged
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #247 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 27 27
Lines 2004 2004
Branches 469 469
=========================================
Hits 2004 2004 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The C11 library rides in the git tags only for now. The sdist include returns with the Python C extension, which needs the sources to build.
jquast
added this pull request to stack #249
September 15, 2026 13:18
Owner
Author
|
Improvement was only up to 1.2x with WIDE and other "hot path" tables, with more tables for diminishing returns, code is too complex, i will however use similar technique to improve grapheme iteration by ~2x |
gcb_of() classified every codepoint with a chain of up to ten sequential binary searches, and GCB_OTHER -- the common ASCII and Latin case -- paid all ten. It is the hottest loop in the library, driven per codepoint by grapheme iteration, clip() and wrap(). The classes are disjoint under that precedence order, so they pack into a single nibble-per-codepoint table and one lookup replaces the chain. update-tables.py picks the page size by measuring, so the table is re-costed on each regeneration. Grapheme iteration is 2-2.4x faster, for 15 KB of rodata. tests/test_gcb_class.c checks the table against the chain it replaces for every codepoint, and against the Extend table, which terminal_override.c relied on directly. Also fixes the Makefile 'test' recipe: $(foreach) joins its expansions with a space, so each test binary was invoked with the following '@echo' as its arguments and only the first header was ever printed.
The version is read from the generated wcwidth_config.h, but the regex required MAJOR.MINOR.PATCH followed immediately by the closing quote. A PEP 440 pre-release such as '0.9.0rc1' did not match, the replace returned the whole #define line unchanged, and project() rejected it -- breaking the cmake build and the ctest CI step.
Add clang_format and clang_format_check tox envs, wired into 'format' and 'lint', and add the check to the CI lint matrix. The hand-written C was never fully formatted, so this reformats it in the same commit. Generated tables under src/tables are excluded from the format targets; their layout comes from the templates, and formatting them would fight every regeneration.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds sub-folder to this project,
libwcwidth/, a C11 implementation of the wcwidth API. This is a release candidate, relased by git tags on github along with matching version of python wcwidth, but not yet included as a Python C extension, this code is not distributed by pypi.#248 creates the C Python "shims" to join the projects together.
Note that the C tables are generated from the Unicode 18.0.0 (draft), 18.0 is expected to release later today, so we intend for these to match before any next python release or git version tag.