Fix vendored ujson crashes with sort_keys and buffer leak on errors; test 3.13/3.14 - #128
Merged
Merged
Conversation
- SortedDict_iterEnd released the sorted items list, which Object_endTypeContext then released again. Every sort_keys=True encode of a dict double-freed it, which segfaults on Python 3.14 (ported from ultrajson#243). - The encoder ignored iterNext returning -1, so dicts with unorderable keys segfaulted on all Python versions. Now the TypeError from the sort is raised and encoding stops (ultrajson#247). - Fix the SortedDict_iterNext error path: it released the borrowed value, passed a NULL key to PyTuple_Pack if key conversion failed, and released the item after PyList_SetItem had stolen it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
python_requires allows up to 3.14, but CI only covered 3.9-3.12, which is how the sort_keys crash on 3.14 went unnoticed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
JSON_EncodeObject returned NULL on error without freeing a heap-allocated output buffer, and objToJSON didn't free it when a Python exception was raised. Each failed dumps() of a large object leaked about 1MB. Backport of upstream ultrajson commit 666d159. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes three problems in the vendored ujson (1.35), all found while testing on Python 3.14. The fixes are backported from upstream ultrajson; re-vendoring current upstream isn't an option for 2.x because it changes output (bytes handling, float formatting, NaN, indent spacing).
Crashes with
sort_keys=TrueSortedDict_iterEndreleased the sorted items list, andObject_endTypeContextthen released it again. Everysrsly.ujson.dumps(d, sort_keys=True)on a dict double-freed it, which segfaults on Python 3.14. Ported from ultrajson#243.iterNextreturning -1, so dicts with keys that can't be sorted (e.g.{1: 1, "a": 2}) segfaulted on every Python version. The sort'sTypeErroris now raised, matching stdlibjson, and encoding stops. Ported from ultrajson#247.SortedDict_iterNexterror path, which released a borrowed value, passed a NULL key toPyTuple_Packif key conversion failed, and released the item afterPyList_SetItemhad taken ownership of it.srsly.json_dumps(sort_keys=True)was never affected, because it falls back to stdlibjson; only directsrsly.ujsoncallers were.Buffer leak on encoding errors
When encoding a large object failed partway through (NaN, unserializable values, unorderable keys), the heap-allocated output buffer was never freed: about 1MB per failed call. Backport of upstream commit 666d159.
CI
Adds Python 3.13 and 3.14 to the test matrix.
python_requiresallows up to 3.14, but CI only covered 3.9–3.12, which is how the 3.14 crash went unnoticed.Tests: new test functions only. Without the fixes they crash (unorderable keys on all versions, everything with
sort_keyson 3.14) or fail the leak check (~105MB over 100 calls).🤖 Generated with Claude Code