feat: Add big endian fixtures with predictor 2 and predictor 3 - #73
Open
james-willis wants to merge 1 commit into
Open
james-willis wants to merge 1 commit into
james-willis wants to merge 1 commit into
Conversation
james-willis
marked this pull request as ready for review
October 1, 2026 21:03
james-willis
force-pushed
the
jw/big-endian-fixtures
branch
from
October 1, 2026 21:43
06cdf41 to
788ee9f
Compare
This was referenced Oct 1, 2026
GDAL's COG driver can't write big endian files, so these are rewritten from a little endian COG through the GTiff driver with COPY_SRC_OVERVIEWS, which keeps the COG layout. - uint16_1band_lzw_block128_predictor2_big_endian: readers must swap samples to native order before undoing horizontal differencing. - float32_1band_deflate_block64_predictor3_big_endian: predictor 3 byte planes are most significant byte first in every file, so readers must not swap these samples. A test checks that both files are MM, carry their predictor on every IFD, and match their little endian source at every level.
james-willis
force-pushed
the
jw/big-endian-fixtures
branch
from
October 1, 2026 22:16
788ee9f to
37b239b
Compare
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The fixtures, generation path, metadata, and regression coverage consistently satisfy the stated endian and predictor requirements.
Review effort: Balanced
Findings: None
What changed in this PR
Adds big-endian predictor fixtures to validate downstream GeoTIFF decoding behavior.
Changes:
- Adds Predictor 2 and Predictor 3 big-endian COG fixtures.
- Adds reusable endian-rewriting logic and fixture validation tests.
- Configures pytest imports for generator modules.
| File | Description |
|---|---|
pyproject.toml |
Adds the repository root to pytest’s Python path. |
rasterio_generated/write_utils.py |
Adds the big-endian COG rewrite helper. |
tests/test_big_endian.py |
Validates byte order, predictors, and pixels. |
rasterio_generated/fixtures/uint16_1band_lzw_block128_predictor2_big_endian.py |
Generates the Predictor 2 fixture. |
rasterio_generated/fixtures/uint16_1band_lzw_block128_predictor2_big_endian.tif |
Provides the generated uint16 fixture. |
rasterio_generated/fixtures/uint16_1band_lzw_block128_predictor2_big_endian_info.md |
Records uint16 fixture metadata. |
rasterio_generated/fixtures/float32_1band_deflate_block64_predictor3_big_endian.py |
Generates the Predictor 3 fixture. |
rasterio_generated/fixtures/float32_1band_deflate_block64_predictor3_big_endian.tif |
Provides the generated float32 fixture. |
rasterio_generated/fixtures/float32_1band_deflate_block64_predictor3_big_endian_info.md |
Records float32 fixture metadata. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two big endian (
MM) fixtures, as requested in developmentseed/deck.gl-raster#687.uint16_1band_lzw_block128_predictor2_big_endianfloat32_1band_deflate_block64_predictor3_big_endianBoth are 128×128 with 64×64 tiles and one overview, and pass
rio cogeo validate.How they're made
GDAL's COG driver has no
ENDIANNESSoption, so a newrewrite_big_endianhelper inwrite_utils.pycopies a little endian COG through the GTiff driver withCOPY_SRC_OVERVIEWS=YESandENDIANNESS=BIG, which keeps the COG layout. The uint16 fixture is rewritten fromuint16_1band_lzw_block128_predictor2; the float32 one from anp.linspaceCOG built the same way asfloat32_1band_lerc_block32.Tiling, compression and predictor are copied from the source, which relies on #74 (
write_cognow writes the predictor into the COG).Tests
tests/test_big_endian.pychecks, for each fixture, that the file starts withMM, that every IFD carries the expected predictor (viatifffile), and that every level matches its little endian source as read by rasterio.pyproject.tomlgainspythonpath = ["."]so the test can import the generators.pixi run checkandpixi run testpass locally, and regenerating is byte-for-byte stable.Downstream check
With deck.gl-raster's fixtures submodule pointed at this branch and both fixtures added to
integration-rasterio.test.ts:tile pixel data matches, and the predictor 3 fixture passes, as expected.