fix(vignette): use pre-cropped fixtures instead of ~2.7 GB of downloads - #35
Merged
eliotmcintire merged 2 commits intoSep 1, 2026
Merged
Conversation
Firesense_LCC_flammability.Rmd downloaded the national Land Cover of Canada 2015 raster (1.9 GB zip) and the National Fire Database polygons (778 MB zip) at vignette-build time, to analyse a 50 km square in central Saskatchewan. Under a full CI matrix, five Ubuntu legs pulled ~13.5 GB concurrently from two NRCan servers. Truncated transfers produced short WKB buffers, which GEOS reported as `ParseException: Unexpected EOF parsing WKB` from inside sf::st_crop() -> st_intersection(). All five Ubuntu legs failed; macOS (1 leg) and Windows (3 legs) passed, which is the concurrency signature rather than a library-version one. The existing HEAD reachability probe could not catch this: the servers are up, the transfer is simply cut short. Removed, along with the macOS `archive` work-around, since nothing is downloaded or extracted any more. Ships two fixtures in inst/extdata (680 KB total): - lcc_studyArea.tif 1359x1977 @ 30 m, exact window extract - firePolygons_studyArea.gpkg 11 features, 2007-2024 The live-download code is preserved verbatim in a commented block so the real workflow stays visible as documentation. Fixes a latent bug while here: NFDB_poly.zip contains TWO layers (1972to2020 and 2021to2024) and sf::read_sf reads only the first, so the live version silently excluded every fire after 2020. The fixture combines both layers before filtering YEAR > 2004, and so covers 2007-2024. `library(reproducible)` dropped from the packages chunk -- prepInputs and checkPath are only used by the commented live block now. Verified: rmarkdown::render() completes with no network access, and the computed burn table is unchanged in structure (10 LCC classes). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the five failing Ubuntu legs on #31.
The failure
Why it happened
The vignette downloaded, at build time:
ftp.maps.canada.ca)cwfis.cfs.nrcan.gc.ca)~2.7 GB per leg, to analyse a 50 km square. With five Ubuntu legs that is ~13.5 GB pulled concurrently from two NRCan servers. When a transfer truncates,
sf::read_sfgets a short shapefile; the first GEOS operation then reads past the end of the WKB buffer and reportsUnexpected EOF. "Unexpected EOF" is a truncated-file symptom, not a library-version one.All five Ubuntu legs failed; macOS (1 leg) and Windows (3 legs) passed. That is the concurrency signature.
The existing HEAD reachability probe cannot catch this — the servers are up, the transfer is simply cut short.
The fix
Two fixtures in
inst/extdata, 680 KB total:lcc_studyArea.tifgdal_translate -projwin, no resampling)firePolygons_studyArea.gpkgThe live-download code is kept verbatim in a commented block, so the real workflow remains visible as documentation.
Latent bug fixed while here
NFDB_poly.zipcontains two layers —NFDB_poly_1972to2020andNFDB_poly_2021to2024— andsf::read_sfreads only the first. The live version therefore silently excluded every fire after 2020, despite filteringYEAR > 2004. The fixture combines both layers, then filters, and covers 2007–2024.How the fixtures were built
GDAL reads these archives without downloading them, via
/vsizip//vsicurl/<url>— it range-requests the ZIP central directory, then fetches only the member bytes it needs. The 778 MB fire archive clipped in 27 s. (The 1.9 GB raster is DEFLATE-compressed inside its zip, so that one degraded to a full download — worth knowing before relying on the technique.)Verified
rmarkdown::render()completes with no network access, and the burn table is unchanged in structure (10 LCC classes, e.g. class 10 at 83% burned).Also drops
library(reproducible)from the packages chunk —prepInputs/checkPathare now only in the commented block.🤖 Generated with Claude Code
https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa