Tracker purpose
Coordinate the repository-isolated ReFrame/BenchExec certification system, dedicated-host execution on OVHC-AGENCY, full Graph500 ladder through S26, parity-based retirement of legacy orchestration, and separate GDC benchmark suites.
Scope
- Keep benchmark orchestration under top-level
benchmarks/ with independent locks.
- Exercise GraphForge only through ordinary supported public APIs/CLI/thin bindings.
- Run the Graph500 ladder and host-local GDC work on OVHC-AGENCY under the
local-linux-cgroups-v2 profile (BenchExec cgroups v2, native local admission).
- Keep scale datasets/projects/exports/imports on this host’s local ext4 NVMe work root; do not use overlay-rooted Cloud Agent VMs or disk-constrained laptops as the SUT.
- Define and execute the ladder
S18 -> S19 -> S20 -> S22 -> S24 -> S25 -> S26 with first-failure stopping and adjacent-rung admission.
- Keep Graph500 and each GDC workload as distinct suites.
- Preserve legacy evidence until accepted parity/cutover.
- Retain disposable Fly adapter code only as optional offline tooling; it is not the close path for S18–S26 certification.
Closure standard
BDD completion scenario
Given a clean checkout, pinned commit, explicit maximum scale, and OVHC-AGENCY with local-linux-cgroups-v2
When an operator selects a Graph500 or GDC profile
Then the OSS harness runs the ordinary public product surface under declared limits
And emits sanitized versioned evidence
And removes all temporary work-root artifacts.
Non-goals
CodSpeed replacement, benchmark-only product hooks, developer-laptop or overlay-VM scale claims, persistent cloud provider infrastructure as the SUT, or universal limits inferred from one host beyond the declared OVHC-AGENCY certification envelope.
Related issues
#745, #900, #901, #904, #951, #956, #958, #959, #991.
Approved release scope and dependencies (2026-09-04)
Assigned to M11 and explicitly included in v0.6.0. Complete all existing GDC/live-suite and parity outcomes before M12 readiness #1096 and M13/#1095 publication. This tracker is a native readiness blocker.
#900 remains this tracker's native completion blocker and supplies the single OVHC-AGENCY Graph500 ladder. #952 must not block #900, which would create a cycle. Reuse that same evidence for #745 and this tracker; no duplicate scale run is required.
Closed child issues alone do not establish completion. Independently verify current code and evidence for outstanding live-suite/parity PRs #1082-#1086 against the existing acceptance criteria. The earlier disposition excluding the benchmark program from the RC is withdrawn. Optional Fly tooling remains outside the approved host close path.
Confirmed benchmark target (2026-09-04)
Maintainer decision: use Graph500-compliant generated data for GraphForge scale tests, clearly labeled. This release does not claim an official Graph500 BFS/SSSP result or TEPS score. GraphForge's ingest, persistence, query, portable interchange, and recovery measurements remain product lifecycle tests.
Preserve the required Graph500 input semantics and publish the pinned generator identity, SCALE, edge factor, seed, raw tuple counts, and GraphForge stored-edge policy. Correct the current generator's unscrambled vertex-label behavior before making the compliant-data claim; existing S18/S19 evidence remains useful engineering diagnosis, not proof of corrected-generator compliance.
Streamline the OVHC-AGENCY path: read-only planning must not prevent execution; admission must use actual work-root free capacity and the declared reserve; native run evidence and cleanup must satisfy consumers without Fly image/provider requirements or hand-entered throughput certificates. Retain declared resource limits, first-failure stopping, real phase resource evidence, correctness/recovery checks, and one reusable ladder result across the existing completion trackers. The observed RSS growth remains owned by #1094; the Graph500 data standard does not define GraphForge's RSS-growth policy.
Tracker purpose
Coordinate the repository-isolated ReFrame/BenchExec certification system, dedicated-host execution on OVHC-AGENCY, full Graph500 ladder through S26, parity-based retirement of legacy orchestration, and separate GDC benchmark suites.
Scope
benchmarks/with independent locks.local-linux-cgroups-v2profile (BenchExec cgroups v2, native local admission).S18 -> S19 -> S20 -> S22 -> S24 -> S25 -> S26with first-failure stopping and adjacent-rung admission.Closure standard
BDD completion scenario
Given a clean checkout, pinned commit, explicit maximum scale, and OVHC-AGENCY with
local-linux-cgroups-v2When an operator selects a Graph500 or GDC profile
Then the OSS harness runs the ordinary public product surface under declared limits
And emits sanitized versioned evidence
And removes all temporary work-root artifacts.
Non-goals
CodSpeed replacement, benchmark-only product hooks, developer-laptop or overlay-VM scale claims, persistent cloud provider infrastructure as the SUT, or universal limits inferred from one host beyond the declared OVHC-AGENCY certification envelope.
Related issues
#745, #900, #901, #904, #951, #956, #958, #959, #991.
Approved release scope and dependencies (2026-09-04)
Assigned to M11 and explicitly included in v0.6.0. Complete all existing GDC/live-suite and parity outcomes before M12 readiness #1096 and M13/#1095 publication. This tracker is a native readiness blocker.
#900 remains this tracker's native completion blocker and supplies the single OVHC-AGENCY Graph500 ladder. #952 must not block #900, which would create a cycle. Reuse that same evidence for #745 and this tracker; no duplicate scale run is required.
Closed child issues alone do not establish completion. Independently verify current code and evidence for outstanding live-suite/parity PRs #1082-#1086 against the existing acceptance criteria. The earlier disposition excluding the benchmark program from the RC is withdrawn. Optional Fly tooling remains outside the approved host close path.
Confirmed benchmark target (2026-09-04)
Maintainer decision: use Graph500-compliant generated data for GraphForge scale tests, clearly labeled. This release does not claim an official Graph500 BFS/SSSP result or TEPS score. GraphForge's ingest, persistence, query, portable interchange, and recovery measurements remain product lifecycle tests.
Preserve the required Graph500 input semantics and publish the pinned generator identity, SCALE, edge factor, seed, raw tuple counts, and GraphForge stored-edge policy. Correct the current generator's unscrambled vertex-label behavior before making the compliant-data claim; existing S18/S19 evidence remains useful engineering diagnosis, not proof of corrected-generator compliance.
Streamline the OVHC-AGENCY path: read-only planning must not prevent execution; admission must use actual work-root free capacity and the declared reserve; native run evidence and cleanup must satisfy consumers without Fly image/provider requirements or hand-entered throughput certificates. Retain declared resource limits, first-failure stopping, real phase resource evidence, correctness/recovery checks, and one reusable ladder result across the existing completion trackers. The observed RSS growth remains owned by #1094; the Graph500 data standard does not define GraphForge's RSS-growth policy.