Add report generator lifecycle tests

[Amaury Levé (Evangelink)]

ReportGeneratorBase owns the session lifecycle shared by every report-generator extension, but that orchestration had no direct unit coverage. This adds focused coverage through CtrfReportGenerator, the smallest concrete implementation, without changing production code.

The tests verify:

• enablement from the --report-ctrf option
• invalid finishing-before-starting lifecycle ordering
• passed and failed test result consumption while ignoring unrelated messages
• report artifact publication with the active session UID
• no output-device writes on the warning-free path

The test harness uses the real internal service provider with mocked platform dependencies. The unreachable-state assertion compares exception full names because the exception polyfill has separate type identities across target assemblies.

Tests: Microsoft.Testing.Extensions.UnitTests (1,495 passed, 7 skipped) and the focused lifecycle suite (5 passed).

Fixes #10850

[github-actions]

🧪 Expert test review — PR #10860

Reviewed the 5 new test methods in CtrfReportGeneratorLifecycleTests.cs (the only test file changed in this PR). All five are well-constructed: clear Arrange-Act-Assert structure, meaningful value/state assertions (not just null checks), correct exception-type assertions with a documented cross-assembly caveat, and correct mock verification (Times.Never) for the no-warning path. No high-confidence actionable findings — nothing warrants an inline suggestion.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CtrfReportGeneratorLifecycleTests. FullLifecycle_ ConsumesTestNodesAndPublishesArtifactAsync	4/4 killed	Parses the emitted JSON and asserts exact test/passed/failed counts, plus session UID and file extension.	—
A (90–100)	new CtrfReportGeneratorLifecycleTests. IsEnabledAsync_ ReturnsFalse_ WhenOptionIsNotSetAsync	1/1 killed	Directly asserts the negative branch of the option-set contract.	—
A (90–100)	new CtrfReportGeneratorLifecycleTests. IsEnabledAsync_ ReturnsTrue_ WhenOptionIsSetAsync	1/1 killed	Directly asserts the positive branch; paired with the false-case test for full boolean coverage.	—
A (90–100)	new CtrfReportGeneratorLifecycleTests. OnTestSessionFinishingAsync_ WithoutPriorStart_ ThrowsUnreachableExceptionAsync	1/1 killed	Exception test with a well-reasoned type-name comparison to avoid cross-assembly type-identity pitfalls on non-NETCOREAPP TFMs.	—
A (90–100)	new CtrfReportGeneratorLifecycleTests. OnTestSessionFinishingAsync_ WithoutWarning_ PublishesArtifactWithoutOutputAsync	1/1 killed	Correctly verifies the output device is never invoked when there is no warning, using a meaningful mock verification.	—

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 75.4 AIC · ⌖ 1.16 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

New unit tests for CtrfReportGenerator lifecycle — clean, well-structured test file that covers enable/disable, the unreachable-exception guard, full lifecycle with artifact publishing and JSON verification, and the no-warning path.

#	Dimension	Verdict
1	Algorithmic Correctness	✅ Clean
2	Error Handling	✅ Clean — UnreachableException assertion uses full-name comparison with a clear comment explaining the cross-assembly polyfill issue
3	Concurrency & Thread Safety	N/A
4	Performance & Allocations	✅ Clean
5	API Design & Public Surface	N/A — test-only code, no public API
6	Binary & Wire Compatibility	N/A
7	Cross-TFM Correctness	✅ Clean — #if NETCOREAPP guard on DisposeAsync
8	Naming & Conventions	✅ Clean — follows Method_Condition_ExpectedAsync naming
9	Assertions & Test Quality	✅ Clean — uses MSTest assertions per BannedSymbols.txt; good use of Assert.HasCount, Assert.EndsWith, JSON structural assertions
10	Resource Management	⚠️ Minor — MemoryFileStream.Dispose() is a no-op; see inline comment
11	Localization	N/A
12	Security	✅ Clean
13	Documentation	✅ Clean — the polyfill comment on the UnreachableException test is helpful
14	Null Safety	✅ Clean — null! for unused IDataProducer parameters is acceptable
15	Code Duplication	⚠️ Minor — MemoryFileStream and TestSessionContextStub are duplicated across multiple test files; see inline comment
16	Backward Compatibility	N/A
17	Configuration	✅ Clean
18	init Accessor Ban	N/A — no public API
19	PublicAPI.Unshipped.txt	N/A
20	StyleCop / EditorConfig	✅ Clean
21	CLI Option Changes	N/A
22	Scope Discipline	✅ Clean — single-concern PR adding tests for one component

Overall: Good test coverage addition. One minor inline comment about MemoryFileStream disposal consistency and potential deduplication — no blockers.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

What changed in this PR

Adds direct unit coverage for shared report-generator lifecycle orchestration through CtrfReportGenerator.

Changes:

• Tests option-based enablement and invalid lifecycle ordering.
• Verifies result consumption, artifact publication, session UID, and warning-free output behavior.

File	Description
test/​UnitTests/​Microsoft.Testing.Extensions.UnitTests/​CtrfReportGeneratorLifecycleTests.cs	Adds five focused lifecycle tests and supporting test doubles.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧵 Parallel-safety audit — PR #10860

Parallelization — audited assembly:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Extensions.UnitTests	MethodLevel ([assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in Program.cs)	CPU count	coverable once the parallel-safety analyzers ship (attribute-based opt-in)

Nothing audited here touches process-global state, shared filesystem paths, or [ResourceLock] / [DoNotParallelize] declarations. Nothing to flag for parallel-safety.

The new file CtrfReportGeneratorLifecycleTests.cs adds 5 [TestMethod]s plus a CreateGenerator helper. Reviewed for category A/B/C/D hazards:

• All mocks (Mock<IFileSystem>, Mock<IEnvironment>, etc.) and the ServiceProvider are created fresh per-test inside CreateGenerator — no shared/static mutable state across tests.
• File I/O is fully virtualized: IFileSystem.NewFileStream returns an in-memory MemoryFileStream (wraps a private MemoryStream), so no real filesystem paths are touched — no shared-path collision is possible.
• No Environment.*, Directory.SetCurrentDirectory, Console.Set*, or culture mutation anywhere in the file.
• No [ResourceLock] / [DoNotParallelize] added, removed, or needed — the tests are naturally isolated per-instance state, so MethodLevel parallelism is safe for this class.

No findings to report.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 37.1 AIC · ⌖ 2.07 AIC · ⊞ 24.8K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧪 Expert test review — PR #10860

Reviewed the 5 new test methods in CtrfReportGeneratorLifecycleTests.cs (added alongside the CTRF report generator lifecycle wiring in ReportGeneratorBase/CtrfReportGenerator).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CtrfReportGeneratorLifecycleTests. FullLifecycle_ ConsumesTestNodesAndPublishesArtifactAsync	4/4 killed	Exercises the full pipeline and asserts summary counts and JSON content, correctly ignoring non-TestNodeUpdateMessage data.	—
A (90–100)	new CtrfReportGeneratorLifecycleTests. IsEnabledAsync_ ReturnsFalse_ WhenOptionIsNotSetAsync	1/1 killed	Complements the true-case test; correctly verifies option-gated enablement.	—
A (90–100)	new CtrfReportGeneratorLifecycleTests. IsEnabledAsync_ ReturnsTrue_ WhenOptionIsSetAsync	1/1 killed	Verifies the option-set path is enabled; behavior is fully captured by the true/false pair.	—
A (90–100)	new CtrfReportGeneratorLifecycleTests. OnTestSessionFinishingAsync_ WithoutPriorStart_ ThrowsUnreachableExceptionAsync	1/1 killed	Correctly exercises the missing-_testStartTime guard; type-name comparison is justified by cross-assembly polyfill identity (documented in-code).	—
A (90–100)	new CtrfReportGeneratorLifecycleTests. OnTestSessionFinishingAsync_ WithoutWarning_ PublishesArtifactWithoutOutputAsync	2/2 killed	Correctly verifies no output device call plus artifact publication when no warning is produced.	—

No new high-confidence issues found. One inline finding (the duplicated MemoryFileStream test helper with an inconsistent silent no-op Dispose()) was already raised and resolved earlier in the PR's review thread; no further action needed there.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 68.1 AIC · ⌖ 1.13 AIC · ⊞ 16.9K · [◷]( · ◷)