Add optional Frontier and BP-Trellis Python batch decoders - #789
Merged
Merged
Conversation
…decoder specs and bindings, propagate provider build exceptions unchanged, and replace the vacuous GIL-release tests
…ovider errors intact, name the failing escalation rung, and show every decoder explicitly in the Guppy DEM workflow
…ta so provider predictions are checked against the model the engines see
…o an oversized request no longer stalls spawning idle threads
…M parse, propagate non-protocol errors from the version probe, and bring the experimental decoder docs in line with the batch API
… draw cannot fail CI
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Frontier and BP-Trellis can now decode Python SampleBatch data and fused DemSampler shots with multiple workers. Their configuration factories and native workers live in the optional pecos-rslib-exp extension:
Both factories expose the underlying decoder configuration, including column ordering and BP-Trellis escalation. Standard pecos.decoders imports work without the experimental package; explicitly importing frontier or bp_trellis from that module lazily loads the optional extension.
The stable extension accepts a versioned Python decoder provider and retains shared batch execution, retries, scoring, and prediction handling. Each worker builds its own native decoder. Native construction and decoding release the GIL, with a Python call boundary per shot. Experimental specs currently support batch and fused sampling APIs, rather than stable DecoderSpec parsing or composition. The publishable pecos-decoders crate has no experimental dependencies.
Configuration is validated once:
TrellisConfig::validate()andBpTrellisConfig::validate()own every rule that does not depend on a DEM, and the spec factories and the directfrom_dem/from_factorsconstructors all call them, so an invalid option raises the sameValueErrorwith the same message everywhere.TrellisOrderingnow lives inpecos-trelliswith aresolve(dem)method and is re-exported frompecos-bp-trellisandpecos-frontier. An exception raised while a provider builds its worker reaches the caller unchanged, with its original type and traceback.The Guppy DEM workflow demonstrates both decoders and requires the optional extension, as its per-shot confidence section already did. The decoder guide documents configuration and the optional imports.
Validation:
just pytest-ci-core: 2338 + 106 + 7475 + 113 passed.pecos-trellis,pecos-bp-trellis, andpecos-frontier.SampleBatch.decode(..., workers=N)now boundsNby the shot count before building its thread pool, matching the boundDemSampler.decodealready applied, so an oversized request no longer spends its time spawning idle threads;workers_usedreports the bounded count. Fixes #790.Replaces #785, which closed when its head branch was renamed.