Skip to content

jpc_t2cod: reject zero shifted sampling factors in packet iterators - #432

Open
iliasabk wants to merge 1 commit into
jasper-software:masterfrom
iliasabk:fix-pi-fpe-shifted-samp
Open

iliasabk wants to merge 1 commit into
jasper-software:masterfrom
iliasabk:fix-pi-fpe-shifted-samp

Conversation

@iliasabk

Copy link
Copy Markdown

Summary

Fixes the floating-point exception reported by OSS-Fuzz in #420 (graphicsmagick:coder_JPC_fuzzer, FPE in jpc_pi_nextrpcl at jpc_t2cod.c:340).

Root cause

In the RPCL/PCRL/CPRL packet iterators, the divisors hsamp << r, vsamp << r, hsamp << rpx and vsamp << rpy are computed in uint_fast32_t. On 32-bit builds (asan_i386) this is 32 bits wide. hsamp/vsamp come from the SIZ marker and may be any value in 1..255 (nonzero is enforced in jpc_cs.c), and the existing overflow check permits r, rpx, rpy up to 29.

For even sampling factors the shift can wrap to zero: e.g. hsamp=8, rpx=29 gives 8 << 29 == 0 (mod 2^32), and x % 0 / JPC_CEILDIV(..., 0) raises SIGFPE. The existing zero checks only cover xstep/ystep, not these per-resolution-level divisors.

Fix

Guard all four shifted divisors before their first use in each of the three affected iterators (jpc_pi_nextrpcl, jpc_pi_nextpcrl, jpc_pi_nextcprl), returning -1 like the neighboring malformed-data checks. In the PCRL/CPRL iterators rpx/rpy are now computed before trx0/try0 so the guard can precede the divisions (no semantic change).

Verification

  • Library builds cleanly (jpc_t2cod.c compiles without warnings).
  • The wrap-to-zero mechanic was verified with the exact operand ranges (8 << 29 == 0, 128 << 25 == 0 for the vsamp path) in 32-bit arithmetic; the new guard fires in both cases.

Disclosure

I am an LLM-based coding agent (Devin). I am available to discuss this change and address review feedback.

The divisors (hsamp << r), (vsamp << r), (hsamp << rpx) and
(vsamp << rpy) used by the RPCL/PCRL/CPRL packet iterators can wrap
around to zero on 32-bit builds. hsamp/vsamp come from the SIZ marker
and may be any value in 1..255, while r/rpx/rpy can reach 29; e.g.
hsamp=8 with rpx=29 yields 8 << 29 == 0 (mod 2^32), causing a division
by zero (SIGFPE) in jpc_pi_nextrpcl as reported by OSS-Fuzz in
jasper-software#420.

Guard all four shifted divisors before their first use, mirroring the
existing zero checks for xstep/ystep.
@iliasabk
iliasabk force-pushed the fix-pi-fpe-shifted-samp branch from bf37982 to 34609b7 Compare September 17, 2026 05:02

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant