Skip to content

Fix three signed-overflow/UB sites in the RAS decoder - #428

Open
iliasabk wants to merge 1 commit into
jasper-software:masterfrom
iliasabk:fix/ras-signed-overflows
Open

iliasabk wants to merge 1 commit into
jasper-software:masterfrom
iliasabk:fix/ras-signed-overflows

Conversation

@iliasabk

Copy link
Copy Markdown

Summary

Fixes #413 — three UBSan sites reachable from jas_image_decode on crafted RAS files:

  1. ras_dec.c:352 — the bit accumulator z is int; (z << 8) overflows into the sign bit for depth >= 24. z is now uint_fast32_t (consumers mask with RAS_ONES anyway).

  2. ras_dec.c:206 — hdr.length = RAS_ROWSIZE(&hdr) * hdr.height overflows int_fast32_t (32-bit on i386/Darwin) for large width * depth * height, e.g. width=8192, depth=32, height=65536 → 32768 * 65536 = 2^31. RAS_ROWSIZE now computes in int_fast64_t, as does the pad computation which also multiplies width * depth.

  3. ras_dec.c:412 — numcolors = 1 << hdr->depth is UB for depth = 32 (checked only after the UB). The bound check now runs on JAS_POW2_X(uint_fast64_t, hdr->depth) before numcolors is materialized; depth <= 8 is required anyway since RAS_CMAP_MAXSIZ is 256.

Testing

No PoC was attached to the issue, so I generated minimal reproducers (32-byte RAS headers + data bytes):

  • depth=32, maptype=EQUALRGB → ras_dec.c:412:17: runtime error: shift exponent 32 is too large
  • type=OLD, 8192x65536x32 → ras_dec.c:206:34: runtime error: signed integer overflow: 32768 * 65536
  • depth=32, data byte 0x80 → ras_dec.c:352:12: runtime error: left shift of 8388608 by 8

All three report cleanly after the patch; malformed files are rejected gracefully.

1) ras_dec.c:352 - the bit accumulator z is a signed int; for depth
   >= 24 the (z << 8) shift overflows into the sign bit (UB). z is
   now uint_fast32_t.

2) ras_dec.c:206 - hdr.length = RAS_ROWSIZE(&hdr) * hdr.height
   overflows 32-bit int_fast32_t for large width*depth*height.
   RAS_ROWSIZE now computes in int_fast64_t, as does the pad
   computation which multiplies width * depth.

3) ras_dec.c:412 - numcolors = 1 << hdr->depth invokes UB for
   depth = 32, and the bound check ran after the UB. Compare the
   bound in uint_fast64_t before materializing numcolors.

Fixes jasper-software#413.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ras_dec.c: 3 signed integer overflows in RAS decoder (UBSan)

1 participant