Stabilize e2e suite under parallelism + add coverage tooling - #146
Merged
Conversation
The Playwright e2e suite passed in isolation but flaked ~7 tests when run in parallel. Root causes (adversarially verified): WASM CPU contention from concurrent @slow HyPhy analyses, and fixed waitForTimeout() races that break under load. The shared-IndexedDB theory was disproven — Playwright isolates IndexedDB per BrowserContext. Changes: - playwright.config.js: cap local workers to 4, retries:1 (safety net only) - package.json: split test:e2e into test:e2e:fast (--grep-invert @slow) and test:e2e:slow (--grep @slow --workers=1) so real-WASM tests run serially while the fast suite stays parallel; total 138 tests preserved, @slow still runs under both chromium and mobile-chrome - helpers.js + specs 06/07/10/14/16: replace fixed waits with web-first assertions (loadDemoFile waits sequence-info; selectMethod asserts value + .method-description; waitForAnalysisCompletion uses expect().toPass()) - 10-branch-selector: fix a real mobile bug (not a flake) — the advanced- options panel overlaps the r=3 SVG node and intercepts pointer events on Pixel 5; scrollIntoViewIfNeeded() + force click after asserting visibility. The old test masked this by silently skipping the click via an if-guard. E2E coverage tooling: monocart-reporter collects V8 coverage over CDP (fixture in e2e/fixtures/coverage.js, env-gated in playwright.config.js); @vitest/coverage-v8 for unit coverage. coverage-e2e/ gitignored. Validated at --retries=0: 7 target tests + branch-selector 72/72, WASM lane 36/36, all green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Playwright e2e suite passed test-by-test in isolation but flaked ~7 tests when run in parallel. This PR makes the suite reliable under parallelism and adds coverage tooling.
Root causes (adversarially verified):
@slowHyPhy analyses (real WASM, ~37s each) running concurrently on one machine stall past timeouts.waitForTimeout()races — hard waits insufficient under load.freshStart()'sdeleteDatabase()can't cross workers. Nosrc/changes were needed for the flakes.Changes
playwright.config.js— cap local workers to 4,retries: 1(safety net only; tests pass atretries:0).package.json— splittest:e2eintotest:e2e:fast(--grep-invert @slow, parallel) andtest:e2e:slow(--grep @slow --workers=1, serial WASM). Total 138 tests preserved (118 fast + 20 slow);@slowstill runs under bothchromiumandmobile-chrome.helpers.js+ specs 06/07/10/14/16 — replace fixed waits with web-first assertions (loadDemoFilewaits onsequence-info;selectMethodasserts value +.method-description;waitForAnalysisCompletionusesexpect().toPass()).10-branch-selector— fixes a real mobile bug (not a flake): on Pixel 5 the advanced-options panel overlaps ther=3SVG tree node and intercepts pointer events, so the click times out.scrollIntoViewIfNeeded()+click({ force: true })after asserting visibility. The old test masked this by silently skipping the click via anif (count > 0)guard.Coverage tooling
monocart-reportercollects V8 coverage over CDP (fixture ine2e/fixtures/coverage.js, env-gated behindE2E_COVERAGE=1). ~72% lines oversrc/.@vitest/coverage-v8(pinned to vitest 3.0.4) for unit coverage.coverage-e2e/gitignored.Validation
All at
--retries=0:10-branch-selector(both projects, ×12): 72/72@slowlane (serial, ×6, both projects): 36/36@slowpresent on mobileNotes
@playwright/testimport was repointed to the coverage fixture; only 06/07/10/14/16 have logic changes.freshStartdoes two page loads and+page.svelteonMountalways boots HyPhy WASM even for non-analysis tests. A single-loadfreshStartwould roughly halve most tests.