geom: validate TWKB ID list count before narrowing to int - #713
Merged
Conversation
parseIDList took its element count as an int, so it needed a numIDs < 0 arm to catch counts that had already wrapped, and its error reported the wrapped negative value rather than the count given in the input. Take the count as a uint64 and bounds-check it before narrowing, matching parsePointArray. The multiplier is 1 rather than the dimension count, since each ID is a single signed varint of at least one byte. The error now reports the true count and the number of remaining bytes. Also cover UnmarshalTWKBIDList, the second exported entry point into this guard, in TestUnmarshalTWKBHugeCount.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aligns TWKB ID-list count validation with the existing point-array guard by accepting the untrusted element count as a uint64, validating it against remaining input bytes before narrowing to int, and improving the resulting error reporting on malformed inputs.
Changes:
- Change
parseIDListto acceptuint64, validate beforeintnarrowing, and improve the “remaining buffer size” error message. - Remove
int(...)casts at theparseIDListcall sites across TWKB collection parsing paths. - Extend
TestUnmarshalTWKBHugeCountto cover theUnmarshalTWKBIDListentry point.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| geom/twkb_parser.go | Updates ID-list count validation and call sites to avoid overflow-before-check and improve error reporting. |
| geom/twkb_test.go | Adds coverage for the UnmarshalTWKBIDList entry point on huge element counts. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The malformed-varint error in nextGeometryCollection said "num polygons" while reading the collection's geometry count. Every other collection parser names its own element type.
The guideline required an entry whenever a change was visible to users of the module, which is stricter than the CHANGELOG's own history: it records internal work that is not externally detectable, and it records nothing about the wording of error messages. State what warrants an entry, note that internal work can be logged, and name the cases too small to bother with.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The TWKB parser has two pre-sized allocations, in
parsePointArrayandparseIDList, and both guard against an element count larger than the remaining input. #712 added those guards but wrote them differently.parsePointArraytakes auint64and checks the count before narrowing it to anint;parseIDListtook anint, so every caller cast the untrusteduvarintat the call site. The guard then needed anumIDs < 0arm to catch a count that had already wrapped, and its error reported the wrapped value rather than the count given in the input:This changes
parseIDListto take auint64and bounds-check it before narrowing, and drops theint(...)cast at the five call sites. The multiplier is 1 rather than the dimension count, since each ID is a single signed varint of at least one byte. The two paths now report the same way:#712 already stopped the panic on both paths. What this changes is the count reported in the ID-list error, and how the two guards are written.
Two smaller changes are included, both from review:
nextGeometryCollectionreportednum polygons varint malformedwhile reading the collection's geometry count, and now reportsnum geometries. Every other collection parser names its own element type. Nothing asserts on the string.The
CLAUDE.mdguideline asking for a CHANGELOG entry "whenever making a change visible to users of this module" was stricter than the CHANGELOG's own history, which records internal work that is not externally detectable and records nothing about the wording of error messages. It now says what warrants an entry and names the cases too small to log.Check List
Have you:
Added unit tests? Yes.
TestUnmarshalTWKBHugeCountgains a case forUnmarshalTWKBIDList, the second exported entry point into this guard, which had no coverage. Thenum geometriesfix changes an error string that no test asserts on, so it adds no test.Add cmprefimpl tests? Not appropriate; cmprefimpl compares value semantics against PostGIS and GEOS, and this only changes error-message text on invalid input.
Updated release notes? No. The Unreleased section of
CHANGELOG.mdalready carries the entry from geom: fix panic in UnmarshalTWKB on huge element counts #712 describing count validation, and both message changes here apply to input that was already rejected. This is the case theCLAUDE.mdchange above now covers explicitly.Updated the README.md? Not applicable; no new functionality.
Related Issue
Benchmark Results