Add optional Frontier and BP-Trellis Python batch decoders

docs/experimental/decoders.md

@@ -1,7 +1,11 @@
 # Experimental decoders
 
-Decoders in `exp/` that are not yet part of the unified `pecos.decoders` surface.
-They are reached through `pecos_rslib_exp` and may change without notice.
+Experimental decoder engines live in `exp/` and may change without notice.
+Frontier and BP-Trellis are available through `pecos_rslib_exp.frontier()` and
+`pecos_rslib_exp.bp_trellis()` for unified parallel batch decoding. Install the
+optional `pecos-rslib-exp` package to use them. Standard decoder imports work
+without this package. The detailed-result APIs described here are also reached
+through `pecos_rslib_exp`.
 
 Two capabilities live here that the production decoders do not offer:
 

docs/user-guide/decoders.md

@@ -49,6 +49,8 @@ The following decoder APIs and supporting types are publicly re-exported from
 | `CheckMatrix` / `SparseMatrix` | Dense or coordinate-form matrix data | Matrix containers used by matching and LDPC decoder constructors. |
 | `MwpmResult` / `BpResult` / `TesseractResult` | Decoder output | Result objects for matching, belief-propagation, and Tesseract decoders. |
 
+The optional factories are described in the [Rust-backed Frontier](#rust-backed-frontier-batch-decoding) and [Rust-backed BP-Trellis](#rust-backed-bp-trellis-batch-decoding) sections below.
+
 Python decoder inputs name their encoding explicitly: use
 `decode_syndrome(...)` for a dense detector vector and
 `decode_from_defects(...)` for sparse detector indices. The BP/LDPC classes'
@@ -331,6 +333,78 @@ match decoder.decode(&syndrome.view()) {
    - Use multiple threads for batch decoding
    - Consider memory layout for cache efficiency
 
+## Rust-backed Frontier batch decoding
+
+Install the optional `pecos-rslib-exp` package for this section and BP-Trellis
+below. Standard `pecos.decoders` imports do not load the experimental extension.
+The explicit `from pecos.decoders import frontier, bp_trellis` convenience import
+loads it lazily and raises an actionable `ImportError` if it is unavailable.
+Experimental factories are excluded from wildcard imports. Their specifications
+work with `SampleBatch.decode(...)` and `DemSampler.decode(...)`; the standard
+`DecoderSpec.parse` strings and composite-spec factories do not load optional
+providers. The experimental calls cross a Python adapter at each shot, with
+native model construction and decoding releasing the GIL.
+
+```python
+from pecos_rslib_exp import frontier
+from pecos_rslib.qec import SampleBatch
+
+dem = "error(0.1) D0 D1 D2 L0\n"
+batch = SampleBatch([[1, 1, 1], [0, 0, 0]], [1, 0])
+result = batch.decode(dem, frontier(k=64), workers=2, predictions=True)
+assert result.predictions == [1, 0]
+assert result.num_errors == 0
+```
+
+Frontier accepts raw DEMs, including hyperedges. `workers=None` selects the
+worker count automatically; `workers=1` runs sequentially. Parallel execution
+releases the Python GIL and uses one Rust decoder per worker, preserving shot
+order. More workers and larger `k` increase memory use.
+
+Options match `pecos_rslib_exp.FrontierDecoder.from_dem`: `k`, `delta`,
+`score_alpha`, `bp_score_iterations`, `column_order`, `merge_indistinguishable`,
+`metric_mode`, and `int_metric_scale`. The default ordering is
+`"deadline_reorder"`; `"time_order"`, `"backward_deadline_reorder"`, and explicit
+column permutations are also accepted. Frontier remains experimental, and
+pruning can make predictions approximate. For per-shot logical masses, pruning
+status, and complementary gaps, use the direct experimental binding.
+
+## Rust-backed BP-Trellis batch decoding
+
+```python
+from pecos_rslib_exp import bp_trellis
+from pecos_rslib.qec import SampleBatch
+
+dem = "error(0.1) D0 D1 D2 L0\n"
+batch = SampleBatch([[1, 1, 1], [0, 0, 0]], [1, 0])
+spec = bp_trellis(
+    k=8,
+    delta=100.0,
+    score_alpha=0.8,
+    bp_score_iterations=5,
+    merge_indistinguishable=True,
+    ordering="deadline",
+    escalation_ks=[32, 128],
+)
+result = batch.decode(dem, spec, workers=2, predictions=True)
+assert result.predictions == [1, 0]
+assert result.num_errors == 0
+```
+
+All seven native BP-Trellis configuration options are exposed. The example opts
+into a retry ladder; the default `escalation_ks=None` disables retries. Retries
+occur only after a no-path result, not after a successful but incorrect prediction.
+Each worker prebuilds its own ladder, increasing construction time and memory.
+`ordering` also accepts `"backward_deadline"`, `"time_order"`, or an explicit
+mechanism permutation. BP-Trellis uses floating-point coset masses and does not
+expose Frontier's integer metric options.
+
+Like Frontier, BP-Trellis accepts raw hyperedges and arbitrary-width observables,
+releases the GIL during batch decoding, and supports automatic worker selection
+and `DemSampler.decode(...)`. It remains experimental. Use
+`pecos_rslib_exp.BpTrellisDecoder` for per-shot confidence, pruning status, and
+retry telemetry; the unified batch result returns predictions and aggregate scores.
+
 ## Hyperedge models and matching decoders
 
 Matching-style decoders (PyMatching, Fusion Blossom and its perturbed
@@ -350,8 +424,8 @@ hyperedges such as bp_osd or tesseract.
 ```
 
 Decode such a model with a decoder that represents hyperedges directly --
-`bp_osd()` or `tesseract()` -- or supply a decomposed projection (a model
-written with `^` separators passes: each component is graphlike). See
+`bp_osd()`, `tesseract()`, `frontier()`, or `bp_trellis()` -- or supply a
+decomposed projection (a model written with `^` separators passes: each component is graphlike). See
 [Experimental Decoders](../experimental/decoders.md) for the Frontier and
 BP-Trellis decoders, which additionally report a per-shot complementary gap,
 and for provenance-based decomposition of a hyperedge model into a graphlike

docs/workflows/guppy-dem-decoding.md

@@ -293,14 +293,17 @@ assert len(sim_shots) == 500
 ## 5. Decode the samples and compute logical error rates
 
 Pass the same sampled batch to `batch.decode(...)` with a typed specification
-for each decoder. A shot counts as a logical error when the predicted
-observable flip disagrees with the flip the sample actually carried. The
+for PyMatching, Tesseract, BP+OSD, and the Rust-backed Frontier and BP-Trellis
+decoders. A shot counts as a logical error when the predicted observable flip
+disagrees with the flip the sample actually carried. The
 returned `DecodeResult` supplies the aggregate count and rate directly.
 
 <!--continuation-->
 ```python
 from pecos.decoders import bp_osd, pymatching, tesseract
 
+from pecos_rslib_exp import bp_trellis, frontier
+
 pymatching_result = batch.decode(
     terminal_graphlike_text,
     pymatching(correlated=True),
@@ -316,22 +319,48 @@ bp_osd_result = batch.decode(
     workers=None,
 )
 
-pymatching_errors = pymatching_result.num_errors
-tesseract_errors = tesseract_result.num_errors
-bp_osd_errors = bp_osd_result.num_errors
-
-shots = batch.num_shots
-assert 0 < pymatching_errors < shots
-assert 0 < tesseract_errors < shots
-assert 0 < bp_osd_errors < shots
+decoder_results = {
+    "pymatching": pymatching_result,
+    "tesseract": tesseract_result,
+    "bp_osd": bp_osd_result,
+}
+experimental_specs = {
+    "frontier": frontier(k=64),
+    "bp_trellis": bp_trellis(k=8, escalation_ks=[32, 128]),
+}
+for name, spec in experimental_specs.items():
+    result = batch.decode(raw_text, spec, workers=4, predictions=True)
+    assert result.execution_path == "parallel"
+    assert result.workers_used == 4
+    assert len(result.predictions) == batch.num_shots
+    decoder_results[name] = result
 
 print("DEM-sampled shots")
-print(f"pymatching  {pymatching_errors:5}   {pymatching_errors / shots:.4%}")
-print(f"tesseract   {tesseract_errors:5}   {tesseract_errors / shots:.4%}")
-print(f"bp_osd      {bp_osd_errors:5}   {bp_osd_errors / shots:.4%}")
-print(f"pymatching execution path: {pymatching_result.execution_path}")
+for name, result in decoder_results.items():
+    assert 0 < result.num_errors < batch.num_shots
+    print(f"{name:11} {result.num_errors:5}   {result.logical_error_rate:.4%}")
+    print(f"{name} execution path: {result.execution_path}")
 ```
 
+Install the optional `pecos-rslib-exp` package to run the Frontier and BP-Trellis
+examples.
+Explicit imports through `pecos.decoders` are also lazy conveniences, but the
+factories and native engines belong to `pecos_rslib_exp`.
+
+`frontier()` uses the native Rust Frontier decoder and accepts the raw DEM,
+including hyperedges. The example decodes shots across four Rust worker threads;
+`predictions=True` also returns observable masks in original shot order. Each
+worker owns a decoder, so memory use grows with the worker count and frontier
+width `k`. Frontier remains experimental; pruning can make its answers approximate.
+
+`bp_trellis()` uses PECOS’s native BP-guided trellis decoder on the same raw
+DEM. Its defaults are `k=8`, `delta=100.0`, `score_alpha=0.8`,
+`bp_score_iterations=5`, `merge_indistinguishable=True`, `ordering="deadline"`,
+and `escalation_ks=None`. The example opts into retry widths `[32, 128]`: these
+are attempted only if the preceding search finds no path. Each worker prebuilds
+the retry decoders, so enabling a ladder increases construction time and memory.
+BP-Trellis also remains experimental; pruning can make predictions approximate.
+
 With `workers=None`, PECOS automatically selects a native-batch, sequential, or
 parallel path based on the decoder and batch size. Pass `workers=N` to request
 an exact worker count. `result.execution_path` reports which path ran. Request
@@ -360,22 +389,25 @@ sim_errors = sim_batch.decode(
 ).num_errors
 
 print(f"simulated shots, pymatching: {sim_errors}/{len(sim_shots)}")
+for name, spec in experimental_specs.items():
+    result = sim_batch.decode(raw_text, spec, workers=4)
+    print(f"simulated shots, {name}: {result.num_errors}/{len(sim_shots)}")
 ```
 
-At this noise level the three decoders land within about a percentage point of
+With the optional package installed, at this noise level the five decoders land within about a percentage point of
 each other on this code; the gaps between decoders widen with code distance and
-with genuinely hyperedge-like noise, which is where BP+OSD and Tesseract consume
-the raw model rather than a graph-like projection.
+with genuinely hyperedge-like noise. Frontier, BP-Trellis, and BP+OSD consume the raw model
+in this example; Tesseract uses the source-informed decomposition chosen above.
 
 ## 6. Optional: per-shot confidence with an experimental decoder
 
 !!! warning "Experimental API"
 
-    The decoders below live in `exp/` and are reached through
-    `pecos_rslib_exp`. They are under active development, are not part of the
-    `pecos.decoders` surface, and may change without notice. They also do not
-    participate in the unified execution planning used above — call them
-    directly rather than through `batch.decode(...)`.
+    The detailed-result APIs below live in `pecos_rslib_exp` and may change
+    without notice. Use `pecos_rslib_exp.frontier()` or `bp_trellis()` with `batch.decode(...)`
+    for parallel predictions and aggregate error rates. Use these direct APIs
+    when you need per-shot confidence data; that data is not returned by
+    `batch.decode(...)`.
 
 Every decoder in stage 5 answers "which observables flipped?". None of them
 reports how close the call was. The experimental Frontier and BP-Trellis
@@ -393,8 +425,8 @@ missing gap is not a pruning signal.
 ```python
 from pecos_rslib_exp import FrontierDecoder
 
-frontier = FrontierDecoder.from_dem(raw_text)
-results = [frontier.decode_syndrome(batch.get_syndrome(shot)) for shot in range(200)]
+frontier_decoder = FrontierDecoder.from_dem(raw_text)
+results = [frontier_decoder.decode_syndrome(batch.get_syndrome(shot)) for shot in range(200)]
 
 assert all(result.status == "exact" for result in results)
 gaps = [result.runner_up_gap for result in results if result.runner_up_gap is not None]
@@ -410,7 +442,6 @@ into a confident majority and a tail worth treating differently:
 <!--continuation-->
 ```python
 confident = [gap for gap in gaps if gap >= 1.0]
-
 print(f"{len(confident)}/{len(gaps)} shots decoded with gap >= 1.0")
 ```
 

exp/pecos-bp-trellis/README.md

@@ -12,3 +12,10 @@ project.
 remain provisional pending broader validation. The shared trellis engine lives
 in `pecos-trellis`; this crate contains PECOS's configuration and decoder
 facade.
+
+The optional `pecos-rslib-exp` package provides `pecos_rslib_exp.bp_trellis(...)`
+for `SampleBatch.decode` and `DemSampler.decode`, including parallel Rust workers.
+All seven configuration options are exposed. Standard `pecos-rslib` and
+`pecos-decoders` do not depend on this crate. The direct
+`pecos_rslib_exp.BpTrellisDecoder` API additionally returns detailed per-shot
+confidence and retry telemetry.

exp/pecos-bp-trellis/src/lib.rs

@@ -27,28 +27,13 @@
 //! trellis engine lives in `pecos-trellis`.
 
 use pecos_decoder_core::ObservableDecoder;
+pub use pecos_trellis::TrellisOrdering;
 use pecos_trellis::{
     DecoderError, MetricMode, ObsMask, SparseDem, TrellisConfig, TrellisDecodeAttempt,
-    TrellisDecoder, TrellisResult, backward_deadline_column_order, deadline_column_order,
+    TrellisDecoder, TrellisResult,
 };
 use std::time::Instant;
 
-/// Processing order used by [`BpTrellisDecoder`].
-#[derive(Clone, Debug, Default, Eq, PartialEq)]
-pub enum TrellisOrdering {
-    /// Compute the deadline-optimized order with [`deadline_column_order`].
-    #[default]
-    Deadline,
-    /// Compute the backward deadline-optimized order with
-    /// [`backward_deadline_column_order`].
-    BackwardDeadline,
-    /// Preserve the detector error model's mechanism order.
-    TimeOrder,
-    /// Use an explicit permutation mapping target positions to source
-    /// mechanism indices.
-    Explicit(Vec<usize>),
-}
-
 /// Configuration for PECOS's [`BpTrellisDecoder`].
 ///
 /// These defaults are provisional. In particular, `k = 8` was validated as
@@ -80,6 +65,42 @@ pub struct BpTrellisConfig {
     pub escalation_ks: Vec<usize>,
 }
 
+impl BpTrellisConfig {
+    /// Validate every rung without building a detector error model.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`DecoderError::InvalidConfiguration`] for an oversized ladder or invalid rung.
+    pub fn validate(&self) -> Result<(), DecoderError> {
+        if u32::try_from(self.escalation_ks.len()).is_err() {
+            return Err(DecoderError::InvalidConfiguration(
+                "escalation ladder has more rungs than escalation_rungs_used can represent".into(),
+            ));
+        }
+        let mut config = self.trellis_config();
+        config.validate()?;
+        for &k in &self.escalation_ks {
+            config.k = k;
+            config.validate()?;
+        }
+        Ok(())
+    }
+
+    fn trellis_config(&self) -> TrellisConfig {
+        TrellisConfig {
+            k: self.k,
+            delta: self.delta,
+            score_alpha: self.score_alpha,
+            column_order: None,
+            merge_indistinguishable: self.merge_indistinguishable,
+            bp_score_iterations: self.bp_score_iterations,
+            // BpTrellis escalation is defined over coset masses, so max-log is deliberately absent from its config.
+            metric_mode: MetricMode::LogSumExpFloat,
+            int_metric_scale: 1024,
+        }
+    }
+}
+
 impl Default for BpTrellisConfig {
     fn default() -> Self {
         Self {
@@ -117,7 +138,7 @@ impl BpTrellisDecoder {
     /// Construct a decoder from a sparse detector error model.
     ///
     /// Unlike [`TrellisDecoder`], the default ordering is the explicitly
-    /// computed [`deadline_column_order`], not input order. Every configured
+    /// computed [`pecos_trellis::deadline_column_order`], not input order. Every configured
     /// escalation rung is constructed here as an independent
     /// [`TrellisDecoder`], so construction cost scales with the full ladder
     /// and decode-time escalation performs no model building.
@@ -127,40 +148,13 @@ impl BpTrellisDecoder {
     /// Returns [`DecoderError`] if ordering generation or the mapped trellis
     /// configuration fails validation.
     pub fn from_sparse_dem(dem: &SparseDem, config: BpTrellisConfig) -> Result<Self, DecoderError> {
+        config.validate()?;
         let build_started = Instant::now();
-        let BpTrellisConfig {
-            k,
-            delta,
-            score_alpha,
-            bp_score_iterations,
-            merge_indistinguishable,
-            ordering,
-            escalation_ks,
-        } = config;
-        if u32::try_from(escalation_ks.len()).is_err() {
-            return Err(DecoderError::InvalidConfiguration(
-                "escalation ladder has more rungs than escalation_rungs_used can represent".into(),
-            ));
-        }
-        let column_order = match ordering {
-            TrellisOrdering::Deadline => Some(deadline_column_order(dem)?),
-            TrellisOrdering::BackwardDeadline => Some(backward_deadline_column_order(dem)?),
-            TrellisOrdering::TimeOrder => None,
-            TrellisOrdering::Explicit(order) => Some(order),
-        };
-        let trellis_config = TrellisConfig {
-            k,
-            delta,
-            score_alpha,
-            column_order,
-            merge_indistinguishable,
-            bp_score_iterations,
-            // BpTrellis escalation is defined over coset masses, so max-log is deliberately absent from its config.
-            metric_mode: MetricMode::LogSumExpFloat,
-            int_metric_scale: 1024,
-        };
+        let mut trellis_config = config.trellis_config();
+        trellis_config.column_order = config.ordering.resolve(dem)?;
         let inner = TrellisDecoder::from_sparse_dem(dem, trellis_config.clone())?;
-        let escalation = escalation_ks
+        let escalation = config
+            .escalation_ks
             .into_iter()
             .map(|rung_k| {
                 TrellisDecoder::from_sparse_dem(