Add optional Frontier and BP-Trellis Python batch decoders

crates/pecos-decoder-core/src/dem.rs

@@ -158,7 +158,9 @@ pub mod utils {
                 // ids; `logical_observable` declares deterministic logicals that
                 // Stim emits with no flipping mechanism but still count.
                 "error" | "logical_observable" => {
-                    for part in &parts[1..] {
+                    // Split on `^` so an unspaced separator counts the same targets
+                    // here as in the other parsers, which tokenize that way.
+                    for part in parts[1..].iter().flat_map(|part| part.split('^')) {
                         if let Some(d_str) = part.strip_prefix('D') {
                             if let Ok(d) = d_str.parse::<usize>() {
                                 max_detector = Some(max_detector.map_or(d, |m: usize| m.max(d)));
@@ -1394,6 +1396,16 @@ mod tests {
         );
     }
 
+    #[test]
+    fn test_metadata_counts_targets_joined_by_an_unspaced_separator() {
+        // Parity with `SparseDem`, whose tokenizer splits on `^`: the highest
+        // detector and observable sit only behind an unspaced separator.
+        let dem = "error(0.1) D0 L0^D3 L5\nerror(0.2) D1 L0\n";
+        let sparse = SparseDem::from_dem_str(dem).unwrap();
+        assert_eq!((sparse.num_detectors, sparse.num_observables), (4, 6));
+        assert_eq!(utils::parse_dem_metadata(dem).unwrap(), (4, 6));
+    }
+
     #[test]
     fn test_non_flattened_dem_rejected() {
         // repeat blocks and shift_detectors would corrupt detector ids if parsed

crates/pecos-decoders/src/batch.rs

@@ -321,6 +321,17 @@ pub fn fused_worker_cap(num_shots: usize) -> usize {
     num_shots.div_ceil(SAMPLING_CHUNK_SHOTS).max(1)
 }
 
+/// Upper bound on useful workers for decoding a sampled batch.
+///
+/// Generic parallel workers pull chunks that shrink down to a single shot, so a
+/// worker beyond one per shot can never be given anything to do. Bounding the
+/// pool here keeps an oversized explicit request from spending its time
+/// spawning idle threads, and `workers_used` reports the bounded count.
+#[must_use]
+pub fn batch_worker_cap(num_shots: usize) -> usize {
+    num_shots.max(1)
+}
+
 /// The single seam constructing the canonical per-chunk RNG for sampling ABI v1.
 ///
 /// Every execution path — sequential, parallel, native — must obtain its chunk
@@ -646,6 +657,14 @@ mod tests {
         assert_eq!(parallel_chunk_shots(5, 0), 5);
     }
 
+    #[test]
+    fn batch_worker_cap_is_one_worker_per_shot() {
+        assert_eq!(batch_worker_cap(0), 1);
+        assert_eq!(batch_worker_cap(1), 1);
+        assert_eq!(batch_worker_cap(2), 2);
+        assert_eq!(batch_worker_cap(100_000), 100_000);
+    }
+
     #[test]
     fn parallel_fake_decoder_reports_lowest_absolute_failure() {
         use pecos_decoder_core::obs_mask::ObsMask;

docs/experimental/decoders.md

@@ -1,7 +1,11 @@
 # Experimental decoders
 
-Decoders in `exp/` that are not yet part of the unified `pecos.decoders` surface.
-They are reached through `pecos_rslib_exp` and may change without notice.
+Experimental decoder engines live in `exp/` and may change without notice.
+Frontier and BP-Trellis are available through `pecos_rslib_exp.frontier()` and
+`pecos_rslib_exp.bp_trellis()` for unified parallel batch decoding. Install the
+optional `pecos-rslib-exp` package to use them. Standard decoder imports work
+without this package. The detailed-result APIs described here are also reached
+through `pecos_rslib_exp`.
 
 Two capabilities live here that the production decoders do not offer:
 
@@ -174,7 +178,9 @@ rest — it has no circuit to re-derive provenance from.
 
 ## What is not here yet
 
-These decoders are not reachable through `DecoderSpec` / `pecos.decoders`, so
-they cannot be passed to `SampleBatch.decode(...)` or `DemSampler.decode(...)`
-and do not participate in unified execution planning, batching, or timing. Use
-their own `decode_syndrome` / `decode_batch` methods directly.
+`frontier()` and `bp_trellis()` are not `DecoderSpec` values: `DecoderSpec.parse`
+does not know their names and the composite factories such as `windowed(...)`
+cannot wrap them. They run through `SampleBatch.decode(...)` and
+`DemSampler.decode(...)` only, and those return predictions and aggregate
+scores; per-shot confidence, pruning status and retry telemetry still come from
+the direct `decode_syndrome` / `decode_batch` methods.

docs/user-guide/decoders.md

@@ -49,6 +49,8 @@ The following decoder APIs and supporting types are publicly re-exported from
 | `CheckMatrix` / `SparseMatrix` | Dense or coordinate-form matrix data | Matrix containers used by matching and LDPC decoder constructors. |
 | `MwpmResult` / `BpResult` / `TesseractResult` | Decoder output | Result objects for matching, belief-propagation, and Tesseract decoders. |
 
+The experimental `frontier()` and `bp_trellis()` factories are not part of this table: they import from `pecos.decoders` only when the optional `pecos-rslib-exp` package is installed, and are described in the [Rust-backed Frontier](#rust-backed-frontier-batch-decoding) and [Rust-backed BP-Trellis](#rust-backed-bp-trellis-batch-decoding) sections below.
+
 Python decoder inputs name their encoding explicitly: use
 `decode_syndrome(...)` for a dense detector vector and
 `decode_from_defects(...)` for sparse detector indices. The BP/LDPC classes'
@@ -331,6 +333,83 @@ match decoder.decode(&syndrome.view()) {
    - Use multiple threads for batch decoding
    - Consider memory layout for cache efficiency
 
+## Rust-backed Frontier batch decoding
+
+Install the optional `pecos-rslib-exp` package for this section and BP-Trellis
+below. Standard `pecos.decoders` imports do not load the experimental extension.
+The explicit `from pecos.decoders import frontier, bp_trellis` convenience import
+loads it lazily and raises an actionable `ImportError` if it is unavailable.
+Experimental factories are excluded from wildcard imports. Their specifications
+work with `SampleBatch.decode(...)` and `DemSampler.decode(...)`; the standard
+`DecoderSpec.parse` strings and composite-spec factories do not load optional
+providers. The experimental calls cross a Python adapter at each shot, with
+native model construction and decoding releasing the GIL.
+
+```python
+from pecos_rslib_exp import frontier
+from pecos_rslib.qec import SampleBatch
+
+dem = "error(0.1) D0 D1 D2 L0\n"
+batch = SampleBatch([[1, 1, 1], [0, 0, 0]], [1, 0])
+result = batch.decode(dem, frontier(k=64), workers=2, predictions=True)
+assert result.predictions == [1, 0]
+assert result.num_errors == 0
+```
+
+Frontier accepts raw DEMs, including hyperedges. `workers=None` selects the
+worker count automatically; `workers=1` runs sequentially. Parallel execution
+releases the Python GIL and preserves shot order. At most one Rust decoder per
+worker is alive at a time, so more workers and larger `k` increase memory use.
+`SampleBatch.decode(...)` builds exactly one decoder per worker;
+`DemSampler.decode(...)` builds one up front to check dimensions and then one per
+scheduled group of sampling chunks, which can be several times the worker count
+on a long run, so a model that is slow to construct pays that cost more than
+once per worker there.
+
+Options match `pecos_rslib_exp.FrontierDecoder.from_dem`: `k`, `delta`,
+`score_alpha`, `bp_score_iterations`, `column_order`, `merge_indistinguishable`,
+`metric_mode`, and `int_metric_scale`. The default ordering is
+`"deadline_reorder"`; `"time_order"`, `"backward_deadline_reorder"`, and explicit
+column permutations are also accepted. Frontier remains experimental, and
+pruning can make predictions approximate. For per-shot logical masses, pruning
+status, and complementary gaps, use the direct experimental binding.
+
+## Rust-backed BP-Trellis batch decoding
+
+```python
+from pecos_rslib_exp import bp_trellis
+from pecos_rslib.qec import SampleBatch
+
+dem = "error(0.1) D0 D1 D2 L0\n"
+batch = SampleBatch([[1, 1, 1], [0, 0, 0]], [1, 0])
+spec = bp_trellis(
+    k=8,
+    delta=100.0,
+    score_alpha=0.8,
+    bp_score_iterations=5,
+    merge_indistinguishable=True,
+    ordering="deadline",
+    escalation_ks=[32, 128],
+)
+result = batch.decode(dem, spec, workers=2, predictions=True)
+assert result.predictions == [1, 0]
+assert result.num_errors == 0
+```
+
+All seven native BP-Trellis configuration options are exposed. The example opts
+into a retry ladder; the default `escalation_ks=None` disables retries. Retries
+occur only after a no-path result, not after a successful but incorrect prediction.
+Each worker prebuilds its own ladder, increasing construction time and memory.
+`ordering` also accepts `"backward_deadline"`, `"time_order"`, or an explicit
+mechanism permutation. BP-Trellis uses floating-point coset masses and does not
+expose Frontier's integer metric options.
+
+Like Frontier, BP-Trellis accepts raw hyperedges and arbitrary-width observables,
+releases the GIL during batch decoding, and supports automatic worker selection
+and `DemSampler.decode(...)`. It remains experimental. Use
+`pecos_rslib_exp.BpTrellisDecoder` for per-shot confidence, pruning status, and
+retry telemetry; the unified batch result returns predictions and aggregate scores.
+
 ## Hyperedge models and matching decoders
 
 Matching-style decoders (PyMatching, Fusion Blossom and its perturbed
@@ -350,8 +429,8 @@ hyperedges such as bp_osd or tesseract.
 ```
 
 Decode such a model with a decoder that represents hyperedges directly --
-`bp_osd()` or `tesseract()` -- or supply a decomposed projection (a model
-written with `^` separators passes: each component is graphlike). See
+`bp_osd()`, `tesseract()`, `frontier()`, or `bp_trellis()` -- or supply a
+decomposed projection (a model written with `^` separators passes: each component is graphlike). See
 [Experimental Decoders](../experimental/decoders.md) for the Frontier and
 BP-Trellis decoders, which additionally report a per-shot complementary gap,
 and for provenance-based decomposition of a hyperedge model into a graphlike

docs/user-guide/dem-from-guppy.md

@@ -242,8 +242,9 @@ assert stats.num_shots == batch.num_shots
 `.logical_error_rate` hold aggregates, optional `.predictions` preserves shot
 order, and optional `.stats` holds timing statistics. With the default
 `workers=None`, the execution planner selects the native-batch, sequential, or
-parallel path. Pass `workers=N` to request a specific worker count and inspect
-`.execution_path` to see which path ran. A former raw-list call such as
+parallel path. Pass `workers=N` to request up to that many workers, never more
+than one per shot, and inspect `.execution_path` and `.workers_used` to see
+which path ran and how many workers it used. A former raw-list call such as
 `detectors, observables = sampler.sample_batch(...)` becomes a batch call
 followed by the two bulk accessors shown above.
 

docs/workflows/guppy-dem-decoding.md

@@ -293,13 +293,14 @@ assert len(sim_shots) == 500
 ## 5. Decode the samples and compute logical error rates
 
 Pass the same sampled batch to `batch.decode(...)` with a typed specification
-for each decoder. A shot counts as a logical error when the predicted
-observable flip disagrees with the flip the sample actually carried. The
+for PyMatching, Tesseract, BP+OSD, and the Rust-backed Frontier and BP-Trellis
+decoders. A shot counts as a logical error when the predicted observable flip
+disagrees with the flip the sample actually carried. The
 returned `DecodeResult` supplies the aggregate count and rate directly.
 
 <!--continuation-->
 ```python
-from pecos.decoders import bp_osd, pymatching, tesseract
+from pecos.decoders import bp_osd, bp_trellis, frontier, pymatching, tesseract
 
 pymatching_result = batch.decode(
     terminal_graphlike_text,
@@ -315,26 +316,62 @@ bp_osd_result = batch.decode(
     bp_osd(max_iter=10, osd_order=1),
     workers=None,
 )
+frontier_result = batch.decode(
+    raw_text,
+    frontier(k=64),
+    workers=4,
+    predictions=True,
+)
+bp_trellis_result = batch.decode(
+    raw_text,
+    bp_trellis(k=8, escalation_ks=[32, 128]),
+    workers=4,
+    predictions=True,
+)
 
-pymatching_errors = pymatching_result.num_errors
-tesseract_errors = tesseract_result.num_errors
-bp_osd_errors = bp_osd_result.num_errors
+assert frontier_result.execution_path == bp_trellis_result.execution_path == "parallel"
+assert frontier_result.workers_used == bp_trellis_result.workers_used == 4
+assert len(frontier_result.predictions) == len(bp_trellis_result.predictions) == batch.num_shots
 
-shots = batch.num_shots
-assert 0 < pymatching_errors < shots
-assert 0 < tesseract_errors < shots
-assert 0 < bp_osd_errors < shots
+decoder_results = {
+    "pymatching": pymatching_result,
+    "tesseract": tesseract_result,
+    "bp_osd": bp_osd_result,
+    "frontier": frontier_result,
+    "bp_trellis": bp_trellis_result,
+}
 
 print("DEM-sampled shots")
-print(f"pymatching  {pymatching_errors:5}   {pymatching_errors / shots:.4%}")
-print(f"tesseract   {tesseract_errors:5}   {tesseract_errors / shots:.4%}")
-print(f"bp_osd      {bp_osd_errors:5}   {bp_osd_errors / shots:.4%}")
-print(f"pymatching execution path: {pymatching_result.execution_path}")
+for name, result in decoder_results.items():
+    assert 0 < result.num_errors < batch.num_shots
+    print(f"{name:11} {result.num_errors:5}   {result.logical_error_rate:.4%}")
+    print(f"{name} execution path: {result.execution_path}")
 ```
 
+`frontier` and `bp_trellis` are experimental: importing them from
+`pecos.decoders` loads the optional `pecos-rslib-exp` package, which must be
+installed, and raises an `ImportError` naming it otherwise. The other three
+factories need nothing beyond the standard install.
+
+`frontier()` uses the native Rust Frontier decoder and accepts the raw DEM,
+including hyperedges. The example decodes shots across four Rust worker threads;
+`predictions=True` also returns observable masks in original shot order. Each
+worker owns a decoder, so memory use grows with the worker count and frontier
+width `k`. Frontier remains experimental; pruning can make its answers approximate.
+
+`bp_trellis()` uses PECOS's native BP-guided trellis decoder on the same raw
+DEM. Its defaults are `k=8`, `delta=100.0`, `score_alpha=0.8`,
+`bp_score_iterations=5`, `merge_indistinguishable=True`, `ordering="deadline"`,
+and `escalation_ks=None`. The example opts into retry widths `[32, 128]`: these
+are attempted only if the preceding search finds no path. Each worker prebuilds
+the retry decoders, so enabling a ladder increases construction time and memory.
+BP-Trellis also remains experimental; pruning can make predictions approximate.
+
 With `workers=None`, PECOS automatically selects a native-batch, sequential, or
 parallel path based on the decoder and batch size. Pass `workers=N` to request
-an exact worker count. `result.execution_path` reports which path ran. Request
+that many workers; a batch with fewer shots than `N` gets one worker per shot
+(at least one), and `result.workers_used` reports the size of the pool that ran.
+`result.execution_path` reports which path ran. Request
 `predictions=True` when you also need each shot's arbitrary-precision
 observable mask; the default avoids materializing them in Python.
 
@@ -354,28 +391,33 @@ sim_batch = SampleBatch(
     [syndrome for syndrome, _ in sim_shots],
     [observable_mask for _, observable_mask in sim_shots],
 )
-sim_errors = sim_batch.decode(
-    terminal_graphlike_text,
-    pymatching(correlated=True),
-).num_errors
-
-print(f"simulated shots, pymatching: {sim_errors}/{len(sim_shots)}")
+sim_results = {
+    "pymatching": sim_batch.decode(terminal_graphlike_text, pymatching(correlated=True)),
+    "tesseract": sim_batch.decode(source_graphlike_text, tesseract(preset="fast", pqlimit=50_000)),
+    "bp_osd": sim_batch.decode(raw_text, bp_osd(max_iter=10, osd_order=1)),
+    "frontier": sim_batch.decode(raw_text, frontier(k=64), workers=4),
+    "bp_trellis": sim_batch.decode(raw_text, bp_trellis(k=8, escalation_ks=[32, 128]), workers=4),
+}
+
+print("simulated shots")
+for name, result in sim_results.items():
+    print(f"{name:11} {result.num_errors:5}/{len(sim_shots)}")
 ```
 
-At this noise level the three decoders land within about a percentage point of
+At this noise level the five decoders land within about a percentage point of
 each other on this code; the gaps between decoders widen with code distance and
-with genuinely hyperedge-like noise, which is where BP+OSD and Tesseract consume
-the raw model rather than a graph-like projection.
+with genuinely hyperedge-like noise. Frontier, BP-Trellis, and BP+OSD consume the raw model
+in this example; Tesseract uses the source-informed decomposition chosen above.
 
 ## 6. Optional: per-shot confidence with an experimental decoder
 
 !!! warning "Experimental API"
 
-    The decoders below live in `exp/` and are reached through
-    `pecos_rslib_exp`. They are under active development, are not part of the
-    `pecos.decoders` surface, and may change without notice. They also do not
-    participate in the unified execution planning used above — call them
-    directly rather than through `batch.decode(...)`.
+    The detailed-result APIs below live in `pecos_rslib_exp` and may change
+    without notice. Use `pecos_rslib_exp.frontier()` or `bp_trellis()` with `batch.decode(...)`
+    for parallel predictions and aggregate error rates. Use these direct APIs
+    when you need per-shot confidence data; that data is not returned by
+    `batch.decode(...)`.
 
 Every decoder in stage 5 answers "which observables flipped?". None of them
 reports how close the call was. The experimental Frontier and BP-Trellis
@@ -393,8 +435,8 @@ missing gap is not a pruning signal.
 ```python
 from pecos_rslib_exp import FrontierDecoder
 
-frontier = FrontierDecoder.from_dem(raw_text)
-results = [frontier.decode_syndrome(batch.get_syndrome(shot)) for shot in range(200)]
+frontier_decoder = FrontierDecoder.from_dem(raw_text)
+results = [frontier_decoder.decode_syndrome(batch.get_syndrome(shot)) for shot in range(200)]
 
 assert all(result.status == "exact" for result in results)
 gaps = [result.runner_up_gap for result in results if result.runner_up_gap is not None]
@@ -410,7 +452,6 @@ into a confident majority and a tail worth treating differently:
 <!--continuation-->
 ```python
 confident = [gap for gap in gaps if gap >= 1.0]
-
 print(f"{len(confident)}/{len(gaps)} shots decoded with gap >= 1.0")
 ```
 

exp/pecos-bp-trellis/README.md

@@ -12,3 +12,10 @@ project.
 remain provisional pending broader validation. The shared trellis engine lives
 in `pecos-trellis`; this crate contains PECOS's configuration and decoder
 facade.
+
+The optional `pecos-rslib-exp` package provides `pecos_rslib_exp.bp_trellis(...)`
+for `SampleBatch.decode` and `DemSampler.decode`, including parallel Rust workers.
+All seven configuration options are exposed. Standard `pecos-rslib` and
+`pecos-decoders` do not depend on this crate. The direct
+`pecos_rslib_exp.BpTrellisDecoder` API additionally returns detailed per-shot
+confidence and retry telemetry.