Skip to content

## Pull request overview #68

Description

@sedv8808

Pull request overview

This PR is a broad robustness + maintenance update to the neotoma2 R package, focusing on more resilient API parsing/transport, new convenience APIs (count(), samples(..., chronname=)), more capable filtering (including spatial loc), and making the test suite more stable via httptest fixtures + CI workflow tuning.

Changes:

  • Added a new exported count() generic with site/sites methods to tally sites, collection units, or datasets.
  • Improved API parsing and request utilities (grouping endpoint responses into a consistent shape, dropping malformed/empty records, Swagger param caching, WKT support for spatial inputs, retry/pagination logic).
  • Reworked tests/CI for reliability (offline fixtures via httptest::with_mock_api(), more CI skips for heavy live calls, workflow timeouts/environment guards).

Reviewed changes

Copilot reviewed 119 out of 128 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
vignettes/neotoma2-package.Rmd Updates vignette to use set_publication() API.
tests/testthat/test_toWide.R Wraps tests with httptest mock API.
tests/testthat/test_taxa.R Skips large live test on CI.
tests/testthat/test_summary.R Adds mocked summary tests + regression for sites with no datasets.
tests/testthat/test_sites.R Adds mocked tests and new count() test; reduces spatial load.
tests/testthat/test_samples.R Adds mocked samples test and CI skips for heavy live tests.
tests/testthat/test_plot.R Avoids interactive device issues by opening a null PDF device.
tests/testthat/test_parseURL.R New unit tests for pagination/retry/base URL/body helpers and swagger caching.
tests/testthat/test_parse_speleothem.R New unit tests for partial speleothem records parsing.
tests/testthat/test_parse_site.R New tests for consistent parsing across endpoints + dataset field retention.
tests/testthat/test_length.R Wraps tests with httptest mock API.
tests/testthat/test_group_data.R New tests for endpoint-shape grouping utilities.
tests/testthat/test_getids.R Wraps tests with httptest mock API.
tests/testthat/test_get_taxa.R Skips very heavy/flaky live integration test on CI and when API unreachable.
tests/testthat/test_get_sites.R Makes heavy spatial/all_data tests CI-safe and uses shared fixtures from setup.
tests/testthat/test_get_manual.R Stubs browser opening in tests and asserts URL.
tests/testthat/test_get_downloads.R Reduces heaviness (limit instead of all_data) + CI skips for flaky calls.
tests/testthat/test_get_documentation.R Stubs browser opening in tests and asserts URL.
tests/testthat/test_get_datasets.R Adds WKT loc coverage + clearer error test; CI skips for spatial flakiness.
tests/testthat/test_filter.R Major expansion of filter tests (multi-condition, spatial, WKT/GeoJSON parity) + split mocked vs live tests.
tests/testthat/test_doi.R Wraps DOI tests with httptest mock API.
tests/testthat/test_datasets.R Wraps datasets accessor tests with httptest mock API and bounds fixture size.
tests/testthat/test_coordinates.R Wraps coordinates tests with httptest mock API.
tests/testthat/test_collunits.R Wraps collunits accessor tests with httptest mock API + adds empty-collunits regression.
tests/testthat/test_clean.R Wraps clean tests with httptest and strengthens duplicate/clean invariants.
tests/testthat/test_cite_data.R Wraps cite_data tests with httptest mock API.
tests/testthat/test_chronologies.R Wraps chronology tests with httptest mock API.
tests/testthat/test_chroncontrols.R Wraps chroncontrols tests with httptest mock API.
tests/testthat/test_c.R Wraps c() tests with httptest mock API.
tests/testthat/test_add_chronology.R Skips heavy Bchron-based live test on CI.
tests/testthat/setup.R Centralizes httptest fixture path + shared Brazil polygon + spatial endpoint probe skip helper.
tests/testthat/fixtures/data_sites24.json Adds hand-made parser fixture.
tests/testthat/fixtures/data_datasets24.json Adds hand-made parser fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites/5.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites/24.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites/24,1001,2001.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites/24,100.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites/101,102.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites/1001.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites/1001,2001,15,24.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites-989a41.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites-8d5f39.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites-6700b2.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/sites-01cad7.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/downloads-711600.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/7870.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/24,7870-c34357.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/24,7870-5b7af1.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/24,100.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/24,100,101.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/24-c34357.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/24-5b7af1.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/101-c34357.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/101-5b7af1.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/1001,2001,15,24.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/100-c34357.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/100-5b7af1.json Adds recorded API fixture.
tests/testthat/fixtures/api.neotomadb.org/v2.0/data/datasets/1,2,3,4,5,6,7,7857,8,7858,10469,9,7859,10-c34357.json Adds recorded API fixture.
README.md Updates release tracking entries and wording.
R/taxon-methods.R Updates roxygen aliases for subset/subscript methods.
R/subset-methods.R Makes datasets/chronologies extractors handle empty/NULL slots safely.
R/speleothems.R Renames speleothem age field usage to rockage.
R/speleothemdetails.R Renames speleothem age field usage to rockage.
R/speleothem-methods.R Renames speleothem age field usage to rockage + roxygen alias updates.
R/site-methods.R Hardens summary() and adds count() methods.
R/set_speleothem.R Renames argument/slot from relativeage to rockage.
R/samples.R Adds chronname support and safer handling of empty datasets.
R/samples-methods.R Updates roxygen aliases for subset/subscript methods.
R/publication-methods.R Updates roxygen aliases for subset/subscript methods.
R/pick.R Adds pick() helper to avoid partial matching and handle endpoint spelling differences.
R/parseLocation.R Adds WKT detection + WKT support path for spatial inputs.
R/parse_speleothem.R Hardens speleothem parsing for missing fields and drops malformed entries.
R/parse_site.R Refactors parsing around grouped endpoint shape; drops malformed entries at each level.
R/group_data.R Replaces endpoint-specific grouping with unified split/group helpers and preserves API order.
R/get_params.R Adds session-level Swagger caching to avoid repeated downloads.
R/filter.R Reworks filter condition analysis and adds spatial loc filter support.
R/dataset-methods.R Updates roxygen aliases for subset/subscript methods.
R/contact-methods.R Updates roxygen aliases for subset/subscript methods.
R/collunits-methods.R Makes c(collunits) robust when all inputs are empty/NULL.
R/clean.R Fixes site de-duplication to operate on site objects directly and handle empty datasets safely.
R/chronology-methods.R Adds missing roxygen alias for [[ method.
R/build_speleothem.R Drops malformed speleothems missing entityid; renames rockage.
R/build_specimen.R Drops malformed specimens missing specimenid.
R/build_site.R Refactors site builder to map API field names in one place and handle missing geometry safely.
R/build_sample.R Adds records_to_df() fast-path and drops malformed samples missing sampleid.
R/build_dataset.R Refactors dataset builder; drops malformed datasets missing datasetid.
R/build_collunit.R Refactors collunit builder; drops malformed collunits missing collectionunitid; uses pick() for spelling variants.
R/build_chron.R Refactors chronology builder; drops malformed chronologies and empty chroncontrols.
R/02_genericDefinitions.R Adds samples(..., chronname=) generic signature and new count() generic.
R/01_classDefinitions.R Renames speleothem slot relativeage -> rockage.
NEWS.md Adds 1.1.0 release notes for new features and robustness changes.
NAMESPACE Exports count() and updates imports for new/refactored code paths.
man/subset.Rd Regenerates docs for updated S4 subset replacement signatures.
man/sub.Rd Regenerates docs for updated S4 subset signatures.
man/sub-subset.Rd Regenerates docs for updated S4 [[<- signatures.
man/sub-sub.Rd Regenerates docs for updated S4 [[ signatures.
man/set_speleothem.Rd Regenerates docs for rockage parameter rename.
man/samples.Rd Documents new chronname argument across samples() methods.
man/samples-sites-method.Rd Documents new chronname argument for samples,sites.
man/plot.Rd Updates plot method signatures/aliases.
man/filter.Rd Documents spatial loc filter support.
man/count.Rd Adds generated documentation for new count() generic/methods.
inst/CITATION Updates author metadata formatting and roles.
DESCRIPTION Bumps version/date and updates Authors@R and Suggests/Imports formatting.
cran-comments.md Adds 1.1.0 CRAN submission notes.
.Rbuildignore Excludes additional non-package files and recorded httptest fixtures from CRAN tarball.
.gitignore Ignores data-raw and vignette build artifacts.
.github/workflows/test-coverage.yml Adds timeouts and API request bounds for coverage job.
.github/workflows/r-cmd-check.yml Modernizes workflow and forces CRAN-like skipping of live API tests.
Suppressed comments (2)

R/site-methods.R:575

  • count(x, level = "datasets") will error when a site has NULL collunits or when a collunit has a NULL/empty datasets slot (cu@datasets@datasets). Given the PR adds multiple guards for optional slots, this should be made NULL-safe too.
    R/site-methods.R:612
  • count() for a single site assumes x@collunits and each cu@datasets slot are always present. This can error on partial objects (e.g. set_site() with no collunits, or collunits with no datasets), even though the rest of the PR treats those slots as optional.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Originally posted by @Copilot in #67 (review)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions