Add acceptance coverage for GH history, step-summary sections, and --show-slowest-tests

[Amaury Levé (Evangelink)]

Why

While reviewing MTP 2.4 readiness we found a few user-facing behaviors that were only exercised by unit tests and had no end-to-end coverage. This adds acceptance tests that drive real Microsoft.Testing.Platform sessions so the behaviors are validated the way users actually hit them.

What

Two areas, both under test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/:

GitHub Actions report (GitHubActionsReportTests.cs)

• New "timed" mode on the existing DummyTestFramework that publishes several passing tests, each carrying a TimingProperty with a distinct duration, so the job summary has real durations to rank.
• --report-gh-step-summary-sections gating: one test asserts slow-tests renders the slowest section while the test-results totals table is gated out, and its inverse asserts test-results keeps the totals table while omitting the slowest section.
• --report-gh-history persistence: runs the same asset twice with distinct GITHUB_RUN_ID values (and a fixed RUNNER_OS so the persisted samples pass schema validation on the next read), then parses the JSON snapshot and asserts the schema version plus two accumulated passed samples, one per run id. This is the first coverage that the history file actually survives and merges across separate runs.

Terminal --show-slowest-tests (ShowSlowestTestsTests.cs, new)

• An active acceptance test (the pre-existing console assertion for this was [Ignore]d). A dedicated asset publishes four passing tests with well-separated durations, and the test asserts the Slowest tests: section is rendered, ranked slowest-first, capped at the requested count (the fastest test is dropped at --show-slowest-tests 3), and absent when the option is not passed.

Notes for reviewers

• No product code changed; this is test-only. OpenTelemetry code and tests were intentionally left untouched (owned by a separate change).
• History determinism relies on distinct GITHUB_RUN_ID per run and a fixed RUNNER_OS; the comment in the test explains why.
• Validated with build.cmd -pack -c Release then running both classes in Release: ShowSlowestTestsTests 6/6 and GitHubActionsReportTests 39/39 passed across net462/net8.0/net10.0.

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

✅ 22/22 dimensions clean — no findings.

Applicable dimensions reviewed in depth (test-only PR, test/IntegrationTests/):

• Algorithmic Correctness: Assertion parameter order verified against Assert.IsGreaterThan(lowerBound, value) / Assert.IsLessThan(upperBound, value) signatures — all correct.
• Test Isolation: History file uses Guid.NewGuid() in TestRunDirectory; ShowSlowestTests uses a shared read-only fixture — no shared mutable state.
• Flakiness Patterns: Durations are fabricated via TimingProperty (no real waits); no duration-string assertions on terminal output; no hard-coded ports or timing dependencies.
• Resource Management: JsonDocument correctly disposed with using var; history file lives in TestRunDirectory (auto-cleaned).
• Assertion Quality: Assertions verify the right things — section presence/absence, ranking order via string indices, history schema structure and sample counts.
• Test Completeness: New features (--report-gh-step-summary-sections, --report-gh-history, --show-slowest-tests) each have positive and negative test cases.

Dimensions N/A (no production code, no analyzers, no IPC, no dependencies, no PowerShell, no MSBuild files): 4, 5, 6, 8, 9, 18, 19, 20, 22.

[github-actions]

🧵 Parallel-safety audit — PR #10863

Parallelization — audited assembly:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Platform.Acceptance.IntegrationTests	MethodLevel	0 (CPU count)	coverable once the parallel-safety analyzers ship (attribute-based [assembly: Parallelize] in Program.cs, unchanged by this PR)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — by severity: Critical 0 · High 0 · Warning 0 · Info 0.

Both changed test files were audited against every category:

• GitHubActionsReportTests.cs — the three new tests (WhenStepSummarySectionsIsSlowTests_OnlyTheSlowestSectionIsRendered, WhenStepSummarySectionsIsTestResults_TheSlowestSectionIsOmitted, WhenHistoryIsEnabled_SamplesPersistAndAccumulateAcrossRuns) and the extended RunAsync helper follow the existing safe pattern: GH_TEST_MODE, GITHUB_RUN_ID, RUNNER_OS, GITHUB_ACTIONS, and GITHUB_STEP_SUMMARY are passed only as environmentVariables into the child test-host process (testHost.ExecuteAsync(..., environmentVariables: ...)), never via Environment.SetEnvironmentVariable in the current process — no process-global mutation. The new history-file path (Path.Combine(TestContext.TestRunDirectory!, $"gh-history-{Guid.NewGuid():N}.json")) and the step-summary path are both per-invocation GUID-suffixed, so there is no cross-test path collision even though these are [DynamicData]-parameterized methods that can run concurrently under MethodLevel.
• ShowSlowestTestsTests.cs (new file) — both test methods only read a shared, already-built, read-only asset (AssetFixture.TargetAssetPath) via TestInfrastructure.TestHost.LocateFrom, the established acceptance-test fixture pattern elsewhere in this project; no writes to shared paths, no static/env/culture/console mutation, and no [ResourceLock]/[DoNotParallelize] declarations added, removed, or mismatched.

The assembly's [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in Program.cs is pre-existing and untouched by this PR, so there is no parallelization-state change to report.

Nothing to flag for parallel-safety.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 80.5 AIC · ⌖ 2 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10863

Reviewed the 5 new test methods added across ShowSlowestTestsTests.cs (new file) and GitHubActionsReportTests.cs (3 new tests + supporting DummyTestFramework "timed" mode). All exercise real, distinguishable production behavior (TestProgressState.GetSlowestTests ranking/ordering, the --report-gh-step-summary-sections gating, and GitHubActionsHistoryStore.Merge cross-run accumulation) with deterministic durations/inputs and assertions that would catch the relevant regressions (wrong sort order, off-by-one count cap, section gating leaking through, sample loss/duplication on merge, wrong schema version). No high-confidence actionable findings identified.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new GitHubActionsReportTests. WhenHistoryIsEnabled_ SamplesPersistAndAccumulateAcrossRuns	5/5 killed	Verifies schema version, sample count, displayName/outcome, and both distinct sorted runIds survive a merge.	—
A (90–100)	new GitHubActionsReportTests. WhenStepSummarySectionsIsSlowTests_ OnlyTheSlowestSectionIsRendered	2/2 killed	Confirms the slow-tests section renders with real data and the totals table is correctly gated out.	—
A (90–100)	new GitHubActionsReportTests. WhenStepSummarySectionsIsTestResults_ TheSlowestSectionIsOmitted	2/2 killed	Mirror of the slow-tests case; confirms section selection excludes the slowest-tests block.	—
A (90–100)	new ShowSlowestTestsTests. WhenEnabled_ RanksTestsSlowestFirstAndHonorsTheRequestedCount	4/4 killed	Deterministic durations plus index-order assertions would catch both a wrong sort and a wrong count cap.	—
A (90–100)	new ShowSlowestTestsTests. WhenNotRequested_ NoSlowestTestsSectionIsRendered	1/1 killed	Confirms the opt-in section is fully absent when the flag is not passed.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 88 AIC · ⌖ 1.16 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

What changed in this PR

Adds end-to-end coverage for MTP’s GitHub Actions reporting and terminal slow-test summary behavior.

Changes:

• Tests GitHub step-summary section filtering and history persistence across runs.
• Tests --show-slowest-tests ranking, count limits, and opt-in behavior.

File	Description
GitHubActionsReportTests.cs	Covers summary filtering, timed tests, and persisted history.
ShowSlowestTestsTests.cs	Adds acceptance coverage for terminal slow-test reporting.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.