Skip to content

test(bench): prove parity and retire legacy scale orchestration #959

Description

@DecisionNerd

Objective

Compare the completed ReFrame/BenchExec/Fly ladder with preserved legacy orchestration and retire only duplicated behavior after semantic parity is demonstrated.

Requirements

  • Preserve exact legacy outcomes as migration fixtures.
  • Use completed test(scale): complete the Fly S18-S26 lifecycle ladder #900 ladder evidence for provider semantics; do not authorize a separate duplicate scale run solely for parity.
  • Compare phases, correctness, failure classes, RSS, storage, logical/physical I/O, reader/publication counters, timing, provenance, and teardown.
  • Run no-cost tiny/local shadow fixtures first.
  • Explain every semantic difference before cutover.
  • Retain bounded correctness tests in product CI.

Acceptance criteria

  • Parity matrix has no unexplained semantic gaps.
  • New harness becomes authoritative only after accepted tiny-fixture and completed-ladder comparison.
  • Removed orchestration has equivalent external coverage.
  • Historical evidence remains readable.
  • No additional S18-S26 provider ladder is run for this issue.

BDD completion scenario

Given identical tiny fixtures, preserved legacy results, and completed #900 evidence
When evidence is normalized and compared
Then semantics agree or every difference is accepted
And duplicated legacy orchestration is retired without losing coverage.

Relationships

Metadata

Metadata

Assignees

No one assigned

    Labels

    testingTest coverage and testing infrastructure

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions