Skip to content

Fix numhprcs*numvprcs overflow defeating numprcs limit - #430

Open
iliasabk wants to merge 1 commit into
jasper-software:masterfrom
iliasabk:fix/numprcs-overflow
Open

iliasabk wants to merge 1 commit into
jasper-software:masterfrom
iliasabk:fix/numprcs-overflow

Conversation

@iliasabk

Copy link
Copy Markdown

Summary

Part of #423 (bug 1) — jpc_dec_tileinit() computes

rlvl->numprcs = rlvl->numhprcs * rlvl->numvprcs;

in unsigned 32 bits and only then applies the 64 * 1024 sanity limit. A crafted codestream whose true product exceeds 2^32 wraps to a small value, passes the check, and leaves band->prcs/prclyrnos severely under-allocated — subsequent packet iteration then reads/writes out of bounds (the issue's ASan report shows the crash in RPCL iteration).

Fix

Evaluate the product in uint_fast64_t and apply the existing 64 * 1024 limit to the true value before assigning numprcs. The post-assignment check is folded into the new pre-assignment check; the debug print still reports the final (bounded) numprcs.

Notes on the other bugs in #423

Bugs 2 (asclen == 0) and 4 (cnt == 0) are already fixed on master (asclen < 1/cnt < 1 checks in jas_icc.c), and the vulnerable code path for bug 3 is inside #if 0 in jp2_dec.c. This PR covers the one remaining live issue.

jpc_dec_tileinit() computes rlvl->numprcs = numhprcs * numvprcs in
unsigned 32 bits and only then applies the 64*1024 sanity limit, so a
crafted codestream whose true product exceeds 2^32 wraps to a small
value, passes the check, and leaves band->prcs/prclyrnos severely
under-allocated. Later packet iteration then reads and writes out of
bounds (see issue jasper-software#423 for an ASan report).

Evaluate the product in uint_fast64_t and apply the limit to the true
value before assigning numprcs. The wrapped-numprcs debug print and
the now-redundant post-assignment check are folded into the new
pre-assignment check.

Part of jasper-software#423 (bug 1).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant