Skip to content

epic(bench): externalize scale certification and add GDC benchmark suites #952

Description

@DecisionNerd

Tracker purpose

Coordinate the repository-isolated ReFrame/BenchExec certification system, dedicated-host execution on OVHC-AGENCY, full Graph500 ladder through S26, parity-based retirement of legacy orchestration, and separate GDC benchmark suites.

Scope

  • Keep benchmark orchestration under top-level benchmarks/ with independent locks.
  • Exercise GraphForge only through ordinary supported public APIs/CLI/thin bindings.
  • Run the Graph500 ladder and host-local GDC work on OVHC-AGENCY under the local-linux-cgroups-v2 profile (BenchExec cgroups v2, native local admission).
  • Keep scale datasets/projects/exports/imports on this host’s local ext4 NVMe work root; do not use overlay-rooted Cloud Agent VMs or disk-constrained laptops as the SUT.
  • Define and execute the ladder S18 -> S19 -> S20 -> S22 -> S24 -> S25 -> S26 with first-failure stopping and adjacent-rung admission.
  • Keep Graph500 and each GDC workload as distinct suites.
  • Preserve legacy evidence until accepted parity/cutover.
  • Retain disposable Fly adapter code only as optional offline tooling; it is not the close path for S18–S26 certification.

Closure standard

  • All native sub-issues are closed or explicitly superseded.
  • Closed profiles, controller, BenchExec authority, and local-host admission path are complete.
  • The Graph500 S18-S26 ladder on OVHC-AGENCY produces normalized reproducible sanitized evidence.
  • S20 fits four hours/4 GiB process RSS; S26 is admitted from S24/S25 below the host free-capacity reserve and completes the billion-edge lifecycle.
  • Local work-root teardown is independently proven.
  • Selected GDC suites validate against pinned upstream identities/references.
  • Shadow parity exists before legacy orchestration is retired.

BDD completion scenario

Given a clean checkout, pinned commit, explicit maximum scale, and OVHC-AGENCY with local-linux-cgroups-v2
When an operator selects a Graph500 or GDC profile
Then the OSS harness runs the ordinary public product surface under declared limits
And emits sanitized versioned evidence
And removes all temporary work-root artifacts.

Non-goals

CodSpeed replacement, benchmark-only product hooks, developer-laptop or overlay-VM scale claims, persistent cloud provider infrastructure as the SUT, or universal limits inferred from one host beyond the declared OVHC-AGENCY certification envelope.

Related issues

#745, #900, #901, #904, #951, #956, #958, #959, #991.

Approved release scope and dependencies (2026-09-04)

Assigned to M11 and explicitly included in v0.6.0. Complete all existing GDC/live-suite and parity outcomes before M12 readiness #1096 and M13/#1095 publication. This tracker is a native readiness blocker.

#900 remains this tracker's native completion blocker and supplies the single OVHC-AGENCY Graph500 ladder. #952 must not block #900, which would create a cycle. Reuse that same evidence for #745 and this tracker; no duplicate scale run is required.

Closed child issues alone do not establish completion. Independently verify current code and evidence for outstanding live-suite/parity PRs #1082-#1086 against the existing acceptance criteria. The earlier disposition excluding the benchmark program from the RC is withdrawn. Optional Fly tooling remains outside the approved host close path.

Confirmed benchmark target (2026-09-04)

Maintainer decision: use Graph500-compliant generated data for GraphForge scale tests, clearly labeled. This release does not claim an official Graph500 BFS/SSSP result or TEPS score. GraphForge's ingest, persistence, query, portable interchange, and recovery measurements remain product lifecycle tests.

Preserve the required Graph500 input semantics and publish the pinned generator identity, SCALE, edge factor, seed, raw tuple counts, and GraphForge stored-edge policy. Correct the current generator's unscrambled vertex-label behavior before making the compliant-data claim; existing S18/S19 evidence remains useful engineering diagnosis, not proof of corrected-generator compliance.

Streamline the OVHC-AGENCY path: read-only planning must not prevent execution; admission must use actual work-root free capacity and the declared reserve; native run evidence and cleanup must satisfy consumers without Fly image/provider requirements or hand-entered throughput certificates. Retain declared resource limits, first-failure stopping, real phase resource evidence, correctness/recovery checks, and one reusable ladder result across the existing completion trackers. The observed RSS growth remains owned by #1094; the Graph500 data standard does not define GraphForge's RSS-growth policy.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testingTest coverage and testing infrastructure

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions