feat(deps): metadata-driven require(extra) dependency guard#339
Merged
Conversation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…etadata directly (#322) CI runs 'mypy src/autointent tests' under strict on py3.10; the test module needed full annotations and patching via the shared importlib.metadata module object (deps.metadata tripped no-implicit-reexport attr-defined). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the import-based
require(dependency, extra=None)guard with a metadata-drivenrequire(extra: str)that reads the installed distribution metadata (importlib.metadata) and verifies every dependency of anautointentextra — recursively, including nested third-party extras liketransformers[torch] → accelerate— is both installed and version-satisfied, raising a single aggregated, actionableImportErrorotherwise.Closes #322.
Why
The old guard had two structural weaknesses:
pyproject.tomlby hand.#322 is the concrete failure:
BertScorercalledrequire("transformers", "transformers"), which passed whenevertransformersimported — even whenaccelerate(pulled bytransformers[torch], needed by HFTrainer) was absent. Users hit a raw, deepImportErrorfrom insideTrainerinstead of our "installautointent[transformers]" message. This is now fixed structurally, with no hand-addedaccelerateguard — the recursion intotransformers' owntorchextra reachesaccelerateautomatically.What changed
src/autointent/_deps.py— resolves an extra's full recursive leaf-requirement set viaimportlib.metadata+packaging, cycle-guarded and memoized, and validates each leaf.pyproject.tomlis not shipped in wheels, but the build bakes the same requirements into the installed metadata, which is read at runtime.require("transformers"),require("peft"),require("sentence-transformers"), …); removed the oldrequire+ its now-unusedimport importlibfrom_utils.py.packagingpromoted to a core dependency (packaging (>=23.2), no upper cap) — it was previously only transitive, which is the same latent-reliance mistake behind bert scorer: opaque ImportError when accelerate is missing (require() guard checks only transformers) #322.user_guides/advanced/02_embedder_configuration.py.Error shape (aggregated, versioned)
Testing
tests/test_deps.py(13 tests) injects a synthetic requirement graph viamonkeypatchso assertions don't depend on what CI installs: all-present (no raise), missing leaf, outdated version, nested-extraacceleratemissing (the #322 regression), cyclic extras terminate, unknown-extraValueError. Plus one real-metadata smoke test for wiring.🤖 Generated with Claude Code