Refactor of diskann-benchmarks to make it potentially easier to spin up new backends

[JordanMaples]

Refactors the diskann-benchmark infrastructure to improve modularity, reduce duplication, and privatize input struct fields behind accessor methods.

• Introduced StreamRunner trait and shared run_streaming helper to unify full-precision and bf-tree streaming benchmark execution, eliminating duplicatedrunbook-driving logic.
• Extracted build_streamer helper for constructing the streaming stack (index + strategy + maintainer).
• Renamed Dynamic → Streaming throughout (e.g., BfTreeDynamicRun → BfTreeStreamingRun, DynamicRunbookParams → StreamingRunbookParams) for clarity.

Input field privatization:

• Made input struct fields private with purpose-specific accessor methods, forcing backends to explicitly handle each field and reject invalidconfigurations.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 92.50936% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.52%. Comparing base (64ac4fb) to head (8af5e9a).

Files with missing lines	Patch %	Lines
diskann-benchmark/src/index/benchmarks.rs	86.00%	7 Missing ⚠️
diskann-benchmark/src/index/streaming/mod.rs	90.54%	7 Missing ⚠️
diskann-benchmark/src/index/streaming/runner.rs	94.93%	4 Missing ⚠️
diskann-benchmark/src/index/inmem/streaming.rs	97.43%	1 Missing ⚠️
diskann-benchmark/src/inputs/graph_index.rs	94.73%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1142   +/-   ##
=======================================
  Coverage   89.51%   89.52%           
=======================================
  Files         487      489    +2     
  Lines       92443    92513   +70     
=======================================
+ Hits        82751    82820   +69     
- Misses       9692     9693    +1     

Flag	Coverage Δ
miri	89.52% <92.50%> (+<0.01%)	⬆️
unittests	89.17% <92.50%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
diskann-benchmark/src/backend/mod.rs	100.00% <ø> (ø)
diskann-benchmark/src/index/build.rs	85.92% <ø> (ø)
diskann-benchmark/src/index/inmem/product.rs	100.00% <100.00%> (ø)
diskann-benchmark/src/index/inmem/scalar.rs	100.00% <100.00%> (ø)
diskann-benchmark/src/index/inmem/spherical.rs	100.00% <100.00%> (ø)
diskann-benchmark/src/index/mod.rs	100.00% <ø> (ø)
diskann-benchmark/src/index/result.rs	49.03% <ø> (ø)
diskann-benchmark/src/index/search/knn.rs	77.77% <ø> (ø)
diskann-benchmark/src/index/search/plugins.rs	73.58% <ø> (ø)
diskann-benchmark/src/index/search/range.rs	0.00% <ø> (ø)
... and 9 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR refactors the diskann-benchmark “graph index” benchmark stack to improve modularity and reduce duplication, primarily by introducing a shared streaming runner/managed layer and by renaming “Dynamic” concepts to “Streaming”. It also restructures benchmark registration so index benchmarks live under a new crate::index module rather than crate::backend::index.

Changes:

• Added a reusable streaming execution path (StreamRunner, Managed, build_streamer, run_streaming) shared across in-mem and bf-tree streaming benchmarks.
• Renamed Dynamic → Streaming for runbook params and streaming runs, updating example inputs and integration tests accordingly.
• Consolidated bf-tree build/stream inputs to support either “none” or “spherical” quantization via a single QuantConfig.

Reviewed changes

Copilot reviewed 36 out of 39 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
diskann-benchmark/src/main.rs	Registers new index benchmark module; updates integration tests and example input filenames.
diskann-benchmark/src/inputs/graph_index.rs	Renames Dynamic→Streaming types; privatizes some input fields and adds accessors; updates tags.
diskann-benchmark/src/inputs/exhaustive.rs	Derives Clone for TransformKind for reuse in new config structures.
diskann-benchmark/src/inputs/bftree.rs	Unifies bf-tree inputs with QuantConfig for build + streaming modes; updates validation/display.
diskann-benchmark/src/index/streaming/stats.rs	Rehomes imports under crate::index::* after refactor.
diskann-benchmark/src/index/streaming/runner.rs	Introduces StreamRunner + Maintainer abstraction used by streaming benchmarks.
diskann-benchmark/src/index/streaming/mod.rs	Adds shared build_streamer and run_streaming helpers for runbook-driven streaming.
diskann-benchmark/src/index/streaming/managed.rs	Adds the Managed ID→slot layer and SlotReclaim policy for streaming backends.
diskann-benchmark/src/index/search/range.rs	Updates imports to new crate::index module structure.
diskann-benchmark/src/index/search/plugins.rs	Updates imports to new crate::index module structure.
diskann-benchmark/src/index/search/mod.rs	New module wrapper for search submodules.
diskann-benchmark/src/index/search/knn.rs	Updates imports to new crate::index module structure.
diskann-benchmark/src/index/result.rs	Updates imports to new crate::index module structure.
diskann-benchmark/src/index/mod.rs	New top-level index module with benchmark registration entrypoint.
diskann-benchmark/src/index/inmem/streaming.rs	Adds in-mem streaming maintenance implementation (DropDeleted + Release).
diskann-benchmark/src/index/inmem/spherical.rs	Adjusts to new module layout and new accessor-based input APIs.
diskann-benchmark/src/index/inmem/scalar.rs	Adjusts to new module layout and new accessor-based input APIs.
diskann-benchmark/src/index/inmem/product.rs	Adjusts to new module layout and new accessor-based input APIs.
diskann-benchmark/src/index/inmem/mod.rs	New inmem module wrapper; exports InmemMaintainer.
diskann-benchmark/src/index/build.rs	New shared build helpers (insert variants, start points, progress meter, save/load).
diskann-benchmark/src/index/bftree/spherical.rs	Refactors bf-tree spherical build benchmark to use QuantConfig and shared quantizer helper.
diskann-benchmark/src/index/bftree/spherical_streaming.rs	Adds bf-tree spherical streaming benchmark using shared streaming helpers.
diskann-benchmark/src/index/bftree/quantizer_util.rs	New shared spherical quantizer construction helper for bf-tree benchmarks.
diskann-benchmark/src/index/bftree/mod.rs	Updates bf-tree benchmark registration and module structure.
diskann-benchmark/src/index/bftree/full_precision.rs	Refactors bf-tree full-precision build benchmark to use unified BfTreeBuild input.
diskann-benchmark/src/index/bftree/full_precision_streaming.rs	Adds bf-tree full-precision streaming benchmark using shared streaming helpers.
diskann-benchmark/src/index/benchmarks.rs	Refactors graph-index benchmark registration and streaming path to use shared helpers.
diskann-benchmark/src/backend/mod.rs	Stops registering the old backend::index benchmarks (now registered via crate::index).
diskann-benchmark/src/backend/index/streaming/mod.rs	Removed (streaming moved under crate::index::streaming).
diskann-benchmark/src/backend/index/streaming/full_precision.rs	Removed (replaced by StreamRunner + inmem maintainer).
diskann-benchmark/src/backend/index/bftree/streaming_utils.rs	Removed (replaced by crate::index::streaming::run_streaming).
diskann-benchmark/src/backend/index/bftree/spherical_streaming.rs	Removed (replaced by crate::index::bftree::spherical_streaming).
diskann-benchmark/src/backend/index/bftree/full_precision_streaming.rs	Removed (replaced by crate::index::bftree::full_precision_streaming).
diskann-benchmark/README.md	Updates example command and job type/tag for streaming runs.
diskann-benchmark/example/graph-index-stream.json	Updates job type to the new streaming tag.
diskann-benchmark/example/graph-index-bftree.json	Updates job type to unified bf-tree tag and adds quantization.kind.
diskann-benchmark/example/graph-index-bftree-stream.json	Updates job type to unified bf-tree streaming tag and adds quantization.kind.
diskann-benchmark/example/graph-index-bftree-stream-spherical.json	New example demonstrating bf-tree streaming with spherical quantization.
diskann-benchmark/example/graph-index-bftree-spherical.json	Updates example to use unified bf-tree tag and nested quantization config.

Comments suppressed due to low confidence (1)

diskann-benchmark/src/inputs/graph_index.rs:1204

• PR description says input fields are privatized behind accessor methods, but StreamingRunbookParams still exposes most fields as pub(crate) and downstream code accesses them directly. Either update the PR description/scope, or make these fields private and add accessors to keep the encapsulation goal consistent.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[magdalendobson]

Thanks Jordan, this looks pretty good to me.

One thing I am wondering is if we're refactoring streaming, there is one other thing in the streaming benchmark that really doesn't make sense, and it's that we don't modify the SearchPhase for streaming when we really ought to. In particular we require a fake groundtruth file, and that file is actually parsed. Furthermore, I'm not sure it really makes sense that we allow an array of thread counts, or maybe also an array of L_search.

Would it make sense to also tackle that as part of a streaming overhaul?

[JordanMaples]

Thanks Jordan, this looks pretty good to me.

One thing I am wondering is if we're refactoring streaming, there is one other thing in the streaming benchmark that really doesn't make sense, and it's that we don't modify the SearchPhase for streaming when we really ought to. In particular we require a fake groundtruth file, and that file is actually parsed. Furthermore, I'm not sure it really makes sense that we allow an array of thread counts, or maybe also an array of L_search.

Would it make sense to also tackle that as part of a streaming overhaul?

Yeah, I can tackle those here.

[hildebrandmw]

Thanks Jordan - this looks like a net simplification, but merging the behavioral changes with the module reorganization is making it hard for me to follow this early on a Monday. Can I request the module reorganization/mechanical changes be moved into their own quick PR? Will review ASAP.