test(vscode): add test-cli lifecycle baseline

[Brian Lam (lambrianmsft)]

Commit Type

• feature - New functionality
• fix - Bug fix
• refactor - Code restructuring without behavior change
• perf - Performance improvement
• docs - Documentation update
• test - Test-related changes
• chore - Maintenance/tooling

Risk Level

• Low - Minor changes, limited scope
• Medium - Moderate changes, some user impact
• High - Major changes, significant user/system impact

What & Why

Adds a latest-stable @vscode/test-cli baseline for the VS Code Logic Apps extension so selected extension-host smoke and generated-workspace lifecycle tests can run independently of ExTester while preserving ExTester for deep webview DOM coverage.

This PR also layers in the ADO Workload Identity Federation auth path needed for complete VS Code E2E coverage in 1ES/MountainPass-compliant Azure DevOps pipelines. The extension now has a provider seam that can use VS Code auth locally and Azure DevOps federated credentials in CI, while preserving existing silent auth, tenant handling, node subscription credential precedence, and the test-cli token fallback.

Impact of Change

• Users: No direct product behavior change; this improves confidence in VS Code extension scenarios on latest stable VS Code and authenticated Azure scenarios.
• Developers: Adds discoverable package scripts, documentation, and helpers for local @vscode/test-cli runs, Azure profile warm-up, generated workspace lifecycle validation, and CI result summaries.
• System: Updates VS Code E2E CI wiring for the new CLI baseline/reporting, adds ADO 1ES/MountainPass-compatible VS Code E2E templates, configures the WIF service connection path, and upgrades @microsoft/vscode-azext-azureauth for Azure DevOps subscription provider support.

Test Plan

• Unit tests added/updated
• E2E tests added/updated
• Manual testing completed
• Tested in: Windows latest-stable VS Code @vscode/test-cli host, local unit/compile validation, YAML parse validation for GitHub and ADO pipeline changes

Validation run locally before integration:

npx biome check --write apps\vs-code-designer\.vscode-test.mjs apps\vs-code-designer\scripts\run-e2e-cli.js apps\vs-code-designer\scripts\summarize-e2e-cli-results.js apps\vs-code-designer\src\test\e2e\cdpClient.ts apps\vs-code-designer\src\test\e2e\workspaceLifecycle.test.ts apps\vs-code-designer\src\test\e2e\README.md
pnpm --dir apps\vs-code-designer run test:e2e-cli:compile
pnpm --dir apps\vs-code-designer run test:extension-unit -- src\app\utils\codeless\__test__\getAuthorizationToken.test.ts --coverage=false
pnpm --dir apps\vs-code-designer run test:e2e-cli:msn-weather-lifecycle -- --visible-delay-ms 15000

Additional validation after integrating ADO/WIF auth and resolving the main merge:

pnpm --dir apps\vs-code-designer run test:extension-unit -- src\app\utils\services\__test__\VSCodeAzureSubscriptionProvider.test.ts src\app\utils\codeless\__test__\getAuthorizationToken.test.ts
pnpm --dir apps\vs-code-designer run test:e2e-cli:compile
pnpm --dir apps\vs-code-designer run test:e2e-cli -- --label unitTests

Results:

• Auth unit tests: 19 passed.
• E2E CLI compile: passed.
• E2E CLI unit label smoke: 12 passed.
• YAML parse validation passed for .github/workflows/vscode-e2e.yml, .azure-pipelines/1esmain.yml, and the new VS Code E2E ADO templates.

Contributors

Brian Lam (@lambrianmsft)

Screenshots/Videos

N/A - test infrastructure, authentication plumbing, and documentation changes only.

[github-actions]

🤖 AI PR Validation Report

PR Review Results

Thank you for your submission! Here's detailed feedback on your PR title and body compliance:

✅ PR Title

• Current: test(vscode): add test-cli lifecycle baseline
• Issue: None — uses a valid test(scope): prefix and is descriptive.
• Recommendation: No changes needed.

---

✅ Commit Type

• Exactly one type selected: test - Test-related changes.
• Appropriate given the change centers on VS Code E2E test infrastructure and CLI test baselines.

---

✅ Risk Level

• Declared High in the body and labeled risk:high — both match, and this matches my advised estimate. The change adds credential/auth handling (ADO Workload Identity Federation, federated service-connection token minting, ARM token acquisition) and upgrades the @microsoft/vscode-azext-azureauth auth dependency in the shipped extension. Per the rubric, security/auth and credential handling changes are High. Correctly declared.

---

✅ What & Why

• Current: Clear explanation of adding a @vscode/test-cli baseline plus the ADO WIF auth provider seam and preserved existing auth behavior.
• Issue: None.
• Recommendation: No changes needed.

---

✅ Impact of Change

• All three audiences addressed (Users, Developers, System) with meaningful detail.
• Recommendation:
  • Users: Adequately described (no direct product behavior change; improved E2E confidence).
  • Developers: Adequately described (new scripts, docs, helpers).
  • System: Adequately described (CI wiring, ADO templates, WIF connection, auth dependency upgrade).

---

✅ Test Plan

• Unit tests updated (getAuthorizationToken.test.ts, VSCodeAzureSubscriptionProvider.test.ts) and E2E tests added (workspaceLifecycle.test.ts), plus manual validation with concrete commands and results. Satisfies CHECK TESTS.

---

✅ Contributors

• Brian Lam (@lambrianmsft) credited. Consider tagging any PMs/designers/reviewers who contributed, but not required.

---

⚠️ Screenshots/Videos

• Marked N/A — test infrastructure, auth plumbing, and documentation only. No user-facing visual UI change detected in the diff, so screenshots are not required.

---

Summary Table

Section	Status	Recommendation
Title	✅	No changes needed
Commit Type	✅	No changes needed
Risk Level	✅	High is correct (auth/credential handling)
What & Why	✅	No changes needed
Impact of Change	✅	No changes needed
Test Plan	✅	No changes needed
Contributors	✅	Optionally tag reviewers/PMs
Screenshots/Videos	⚠️	Not required (no visual change)

---

All required checks pass — this PR is compliant and cleared to merge.

---

Powered by: Copilot CLI (claude-opus-4.8) | Last updated: Sun, 27 Sep 2026 20:56:42 GMT

[Copilot]

Note

Copilot was unable to run its full agentic suite in this review.

Pull request overview

Adds a latest-stable @vscode/test-cli E2E baseline for the VS Code Logic Apps extension, enabling Create Workspace and select lifecycle tests to run independently of ExTester while improving CI visibility.

Changes:

• Added root and app-level pnpm scripts plus Node wrappers to run label-based CLI E2E and multi-host lifecycle flows.
• Introduced new CLI E2E test utilities (dialog guard, CDP helpers, screenshots) and expanded smoke / Create Workspace coverage.
• Updated CI (vscode-e2e.yml) to run a Create Workspace CLI matrix with structured results + aggregate reporting.

Reviewed changes

Copilot reviewed 33 out of 44 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
package.json	Adds root-level forwarding scripts for new CLI E2E entry points.
libs/vscode-extension/src/graphify-out/GRAPH_REPORT.md	Updates generated graph report snapshot metadata/content.
libs/data-mapper-v2/src/graphify-out/GRAPH_REPORT.md	Updates generated graph report snapshot metadata/content.
libs/chatbot/src/graphify-out/GRAPH_REPORT.md	Updates generated graph report snapshot metadata/content.
libs/a2a-core/src/graphify-out/GRAPH_REPORT.md	Updates generated graph report snapshot metadata/content.
apps/vs-code-designer/src/test/e2e/workspaceArtifacts.ts	Adds helpers/assertions for generated workspace artifacts (tasks/launch/settings/csproj).
apps/vs-code-designer/src/test/e2e/webviewTabs.ts	Adds helpers to find/wait/close webview tabs in VS Code.
apps/vs-code-designer/src/test/e2e/visibleDelay.ts	Adds optional “keep host visible” delay for local debug runs.
apps/vs-code-designer/src/test/e2e/testUtils.ts	Adds small string/path helper utilities for tests.
apps/vs-code-designer/src/test/e2e/screenshot.ts	Adds screenshot capture utilities (Windows screen + CDP).
apps/vs-code-designer/src/test/e2e/extension.test.ts	Expands activation smoke to assert dist loading, dependencies, empty startup, and dialog suppression.
apps/vs-code-designer/src/test/e2e/dialogGuard.ts	Adds dialog interception/allowlist to fail tests on unexpected prompts.
apps/vs-code-designer/src/test/e2e/createWorkspaceTypes.ts	Adds typed models for Create Workspace test cases.
apps/vs-code-designer/src/test/e2e/createWorkspaceParityMap.md	Documents ExTester↔CLI parity mapping for Create Workspace + lifecycle coverage.
apps/vs-code-designer/src/test/e2e/createWorkspaceCases.ts	Adds curated Create Workspace case generators and label filtering.
apps/vs-code-designer/src/test/e2e/commands.test.ts	Replaces generic command listing with explicit Logic Apps command/config assertions + dialog guard.
apps/vs-code-designer/src/test/e2e/cdpFormHelpers.ts	Adds CDP-based Create Workspace webview interaction helpers (fields, dropdowns, radios).
apps/vs-code-designer/src/test/e2e/cdpClient.ts	Adds a lightweight CDP client to connect to VS Code’s remote debugging targets.
apps/vs-code-designer/src/test/e2e/azureAuthWarmup.test.ts	Adds Azure auth warm-up test for durable local profiles.
apps/vs-code-designer/src/test/e2e/README.md	Major documentation expansion for running CLI E2E locally and in CI.
apps/vs-code-designer/src/main.ts	Adds env-gated minimal activation and env-gated workspace ensure skip for CLI test runs.
apps/vs-code-designer/src/app/utils/codeless/getAuthorizationToken.ts	Adds test-gated env token fallback for silent auth; fixes return type to allow undefined.
apps/vs-code-designer/src/app/utils/codeless/test/getAuthorizationToken.test.ts	Adds unit coverage for the new test-gated Azure CLI token fallback behavior.
apps/vs-code-designer/scripts/summarize-e2e-cli-results.js	Adds Mocha log parsing + JSON/JUnit/summary generation and aggregate reporting.
apps/vs-code-designer/scripts/run-e2e-cli.js	Adds wrapper to run labels/lifecycle flows, filter noise, enforce “no dialogs”, and clean temp workspaces.
apps/vs-code-designer/scripts/open-e2e-cli-vscode.js	Adds helper to open latest-stable VS Code with an isolated profile for interactive verification/auth.
apps/vs-code-designer/package.json	Replaces direct vscode-test call with wrapper scripts for CLI E2E and lifecycle flows.
apps/vs-code-designer/.vscode-test.mjs	Expands @vscode/test-cli config to multiple labels, env forwarding, and per-run user-data settings.
.squad/knowledge/vscode-e2e-testing.md	Documents learnings/patterns for CLI Create Workspace CI structured results and parity scripts.
.github/workflows/vscode-e2e.yml	Adds Create Workspace CLI matrix job + aggregate report job and wires into overall summary.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.