Use Python extension with libwcwidth - #248
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #248 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 27 27
Lines 2008 2040 +32
Branches 468 473 +5
=========================================
+ Hits 2008 2040 +32 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will improve performance by ×9.9
|
fceb715 to
8056323
Compare
712191d to
36f165b
Compare
36f165b to
edac376
Compare
8056323 to
5899da0
Compare
Adds sub-folder to this project, ``libwcwidth/``, a C11 implementation of the wcwidth API. This is a **release candidate**, relased by git tags on github along with matching version of python wcwidth, but **not yet included as a Python C extension**, this code is not distributed by pypi. #248 creates the C Python "shims" to join the projects together. Note that the C tables are generated from the **Unicode 18.0.0 (draft)**, 18.0 is expected to release later today, so we intend for these to match before any next python release or git version tag.
edac376 to
2e6190b
Compare
5899da0 to
dd83e78
Compare
We expect Unicode 18.0 to release today. - Match new 18.0.0 ``UAX #20`` rules on grapheme clustering InCB/Indic_Conjunct_Break - Stop using python's ``iter_graphemes()``, reduces performance! fixed in next release #248 - parse names from unicode.org data instead of current python - `wcstwidth()` correction table updated with latest ucs-detect results (contour and rio are better-conforming) - skip "bad results" from ucs-detect (delta_y movement in CPR, iTerm2, from correction tables) - correction table file hashes are now deterministic (don't use repr() for hashing -- write our own)
dd83e78 to
6bbbba7
Compare
225afb7 to
8bd4e2d
Compare
d830610 to
84dbc43
Compare
This will merge and release after release of wcwidth 0.8.4. The 0.9.0 release should represent only the change of using C11 libwcwidth as optional Python extension. This PR makes no logical change to wcwidth library, only that it drops Python 3.8 support and improves performance by use of the optional build_ext. This creates a new release process, documented in docs/developing.rst. tox -e check_release calls bin/check-release.py to help ensure all wheels are included. clip() keeps both Python implementations: _clip_simple() for text without cursor movement, and _clip_painter() when movement must be resolved. Retiring _clip_simple() in favor of a single painter cost 20-35% on seven clip() benchmarks, and the extension does not recover it: libwcwidth declines OSC 8 and OSC 66, and is never consulted for control_codes != 'parse', so those inputs are answered by Python in either build. tests/test_clip.py compares the two paths over the parity and fuzz corpora through the public overtyping= parameter. A lot of care was done to make it safe to release to general public, that it should be possible to install from sdist even if compilation fails or is unavailable and gracefully fall back to pure python. cibuildwheel is used to build binaries, and, because we distribute a "py3-none-any" (any os, any architecture), that, if no build wheel exists, then package installs will use the python-only wheel without any error or warning.
84dbc43 to
b36fc0c
Compare
because dependabot is annoying as shit.
…wheels # Conflicts: # docs/libwcwidth.rst # wcwidth/__init__.py
# Conflicts: # docs/libwcwidth.rst # wcwidth/__init__.py
We won't really know until we cut one
This prepares a 0.9.0 release, which only enables use of C11 libwcwidth as optional Python extension. This PR makes no logical changes to the python library behavior except to improve performance ~10-58x by use of the optional
build_ext.This creates a new release process, documented in docs/developing.rst.
tox -e check_releasecallsbin/check-release.pyto help ensure all wheels are included.A lot of care was done to make it safe to release to general public, that it should be possible to install from sdist even if compilation fails or is unavailable and gracefully fallback to pure python. cibuildwheel is used to build binaries, and, because we distribute a "python3-none-any" (any os, any architecture), that, if no build wheel exists, then package installs will use the python-only wheel without any error or warning.