Skip to content

Add optional Frontier and BP-Trellis Python batch decoders - #789

Merged
ciaranra merged 10 commits into
devfrom
trellis-python-batch-decoding
Sep 18, 2026
Merged

ciaranra merged 10 commits into
devfrom
trellis-python-batch-decoding

Conversation

@ciaranra

@ciaranra ciaranra commented Sep 17, 2026

Copy link
Copy Markdown
Member

Frontier and BP-Trellis can now decode Python SampleBatch data and fused DemSampler shots with multiple workers. Their configuration factories and native workers live in the optional pecos-rslib-exp extension:

from pecos_rslib_exp import frontier, bp_trellis

result = batch.decode(dem_text, frontier(k=64), workers=4)

Both factories expose the underlying decoder configuration, including column ordering and BP-Trellis escalation. Standard pecos.decoders imports work without the experimental package; explicitly importing frontier or bp_trellis from that module lazily loads the optional extension.

The stable extension accepts a versioned Python decoder provider and retains shared batch execution, retries, scoring, and prediction handling. Each worker builds its own native decoder. Native construction and decoding release the GIL, with a Python call boundary per shot. Experimental specs currently support batch and fused sampling APIs, rather than stable DecoderSpec parsing or composition. The publishable pecos-decoders crate has no experimental dependencies.

Configuration is validated once: TrellisConfig::validate() and BpTrellisConfig::validate() own every rule that does not depend on a DEM, and the spec factories and the direct from_dem / from_factors constructors all call them, so an invalid option raises the same ValueError with the same message everywhere. TrellisOrdering now lives in pecos-trellis with a resolve(dem) method and is re-exported from pecos-bp-trellis and pecos-frontier. An exception raised while a provider builds its worker reaches the caller unchanged, with its original type and traceback.

The Guppy DEM workflow demonstrates both decoders and requires the optional extension, as its per-shot confidence section already did. The decoder guide documents configuration and the optional imports.

Validation:

  • just pytest-ci-core: 2338 + 106 + 7475 + 113 passed.
  • 94 Rust tests passed across pecos-trellis, pecos-bp-trellis, and pecos-frontier.
  • The GIL-release tests time a single native build call and a single native decode call while a Python thread records its longest stall. Removing the GIL release from either call fails exactly the matching tests.
  • A barrier-based test confirms that the provider bridge runs the requested number of workers concurrently.
  • Worker scaling on a distance-5 circuit-level surface-code DEM (4000 shots, 1 to 16 workers): Frontier 10.9x, BP-Trellis 8.1x, against 7.8x for the built-in BP+OSD specification.
  • Clippy with all targets and warnings denied on every crate that depends on the changed crates; rustfmt, pre-commit, and the Python lint recipe passed.

SampleBatch.decode(..., workers=N) now bounds N by the shot count before building its thread pool, matching the bound DemSampler.decode already applied, so an oversized request no longer spends its time spawning idle threads; workers_used reports the bounded count. Fixes #790.

Replaces #785, which closed when its head branch was renamed.

…ovider errors intact, name the failing escalation rung, and show every decoder explicitly in the Guppy DEM workflow
…ta so provider predictions are checked against the model the engines see
…o an oversized request no longer stalls spawning idle threads
…M parse, propagate non-protocol errors from the version probe, and bring the experimental decoder docs in line with the batch API
@ciaranra
ciaranra merged commit 293b809 into dev Sep 18, 2026
94 checks passed
@ciaranra
ciaranra deleted the trellis-python-batch-decoding branch September 18, 2026 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SampleBatch.decode does not bound an explicit worker count, so a large value stalls in thread-pool construction

1 participant