Skip to content

Fix shift-exponent UB in bitstoint() - #426

Open
iliasabk wants to merge 1 commit into
jasper-software:masterfrom
iliasabk:fix/bitstoint-shift-ub
Open

iliasabk wants to merge 1 commit into
jasper-software:masterfrom
iliasabk:fix/bitstoint-shift-ub

Conversation

@iliasabk

Copy link
Copy Markdown

Summary

Fixes #415 — bitstoint() invokes undefined behavior for a component precision of 32:

  • 1 << (prec - 1) = 1 << 31 overflows into the sign bit of a 32-bit int
  • 1 << prec = 1 << 32 has an out-of-range shift exponent
  • v - (1 << prec) then wraps in unsigned 32-bit, producing a wrong sign extension, and JAS_ONES(32) = (1 << 32) - 1 has the same UB — also reached via inttobits()

Fix

  • Mask computation uses JAS_POW2_X(uint_fast64_t, prec) - 1 (defined for prec <= 32)
  • The sign test uses JAS_POW2_X(uint_fast32_t, prec - 1) (bit 31 is fine in unsigned)
  • The sign extension computes v - 2^prec in jas_seqent_t (64-bit), giving the mathematically correct negative result

Testing

Reproduced with the PoC attached to #415 under UBSan:

jas_image.c:1044:4: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
jas_image.c:1044:4: runtime error: signed integer overflow: -2147483648 - 1

After the patch: no sanitizer report, decode fails gracefully on the malformed input.

For a component precision of 32, the expressions 1 << (prec - 1) and
1 << prec in bitstoint() invoke undefined behavior on 32-bit int, and
JAS_ONES(prec) has the same problem. Compute the masks and the sign
extension in uint_fast64_t/jas_seqent_t instead. inttobits() gets the
same mask fix since JAS_ONES(32) is equally affected.

Fixes jasper-software#415.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

jas_image.c: shift exponent UB in bitstoint() (followup to inttobits fix)

1 participant