ci(e2e): turn on the shared E2E Tests (Playwright) job - #378
Open
rubenvdlinde wants to merge 1 commit into
Open
Conversation
The shared workflow's `E2E Tests (Playwright)` job was never enabled on decidesk, and the state it sat in reads on the PR page exactly like a pass: `if: inputs.enable-playwright && …` evaluates false, so the job reports `skipped` rather than `failure`. 28 spec files / 171 tests were in the repo and none of them had ever run in CI. Two blockers, both fixed here: 1. `additional-apps` was COMMENTED OUT entirely, so no CI instance ever had OpenRegister. That is not an optional dependency for decidesk — `appinfo/routes.php` returns `\OCA\OpenRegister\AppHost\Routes::standard()`, the whole domain model lives in `lib/Settings/decidesk_register.json` plus the 24 `register.d/` fragments, and the SPA reads and writes every entity through `/apps/openregister/api/objects/decidesk/<schema>`. Now pinned to `development` (not `main`) so the CI instance matches the environment the app is actually developed against. 2. The register still would not provision itself. `occ app:enable decidesk` runs its repair step with NO user session, OpenRegister's RBAC denies the import outright, and the step catches \Throwable and downgrades it to a warning — so `occ app:enable` exits 0 with no register. `tests/e2e/ci-seed.sh` therefore does the import explicitly over the admin HTTP API (`POST /apps/decidesk/api/settings/load` → `loadConfiguration(force: true)`, the only path that also merges the `register.d/` fragments), then VERIFIES the register slug and 18 schema slugs read verbatim out of `decidesk_register.json`, probes four object collections for readability, and finally gates on the webpack bundle SERVING as JavaScript. New artefacts: - `tests/e2e/playwright.config.ts` — a CI-only config declaring exactly ONE project. The workflow's run step passes no `--project`, so whichever config it picks runs every project in it; the root config declares `docs-capture` (which would re-shoot every documentation screenshot on each PR) and `visual` (whose own README records that a CI Linux runner cannot byte-match a dev-container PNG baseline). `playwright-test-path: tests/e2e` makes the workflow's first config lookup hit this file instead. The root config is deliberately untouched — the separate `Journeydoc Capture` job greps it for `name: 'docs-capture'`. - `tests/e2e/base-url.ts` — one resolver, replacing 22 copies of `process.env.NEXTCLOUD_URL || 'http://localhost:8080'`. It accepts `PLAYWRIGHT_BASE_URL | NEXTCLOUD_URL | NC_BASE_URL | BASE_URL` (`BASE_URL` is the name the shared workflow actually exports), falls back to `http://localhost:8080` only on CI, and THROWS otherwise. The old default was the shared dev container, and these specs seed real governance objects through the OpenRegister object API. - `tests/e2e/.gitignore` — committed before the artefacts can appear. Anchored with a leading slash so the `visual` project's committed PNG baselines under `tests/e2e/visual/*.spec.ts-snapshots/` are not matched. Controls run locally: the CI config collects 171 tests in 28 files with zero `visual/` or `docs-screenshots` entries, and the resolver raises with the env cleared.
rubenvdlinde
requested review from
Rem-Dam,
SudoThijn,
WilcoLouwerse,
bbrands02,
remko48 and
rjzondervan
as code owners
August 3, 2026 22:22
Contributor
Quality Report — ConductionNL/decidesk @
|
| Check | PHP | Vue | Security | License | Tests |
|---|---|---|---|---|---|
| lint | ✅ | ||||
| phpcs | ✅ | ||||
| phpmd | ✅ | ||||
| psalm | ✅ | ||||
| phpstan | ✅ | ||||
| phpmetrics | ✅ | ||||
| eslint | ✅ | ||||
| stylelint | ✅ | ||||
| build | ✅ | ||||
| composer | ✅ | ✅ 100/100 | |||
| npm | ✅ | ✅ 549/549 | |||
| PHPUnit | ❌ | ||||
| Newman | ❌ | ||||
| Playwright | ❌ |
Quality workflow — 2026-08-03 23:11 UTC
Download the full PDF report from the workflow artifacts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Turns on the shared workflow's
E2E Tests (Playwright)job for decidesk, and adds the four artefacts it needs to be genuinely green rather than green-by-absence.Why it was not running
The job's gate is
if: inputs.enable-playwright && !cancelled() && needs.security.result != 'failure'.enable-playwrightwas never set, so the job reportedskipped— which on the PR page is visually indistinguishable from a pass. 28 spec files / 171 tests were sitting intests/e2e/and none of them had ever executed in CI.Turning the flag on alone would not have worked, for two independent reasons:
additional-appswas commented out entirely. OpenRegister is not an optional dependency for decidesk —appinfo/routes.phpreturns\OCA\OpenRegister\AppHost\Routes::standard(), the whole domain model lives inlib/Settings/decidesk_register.jsonplus 24register.d/fragments, and the SPA reads and writes every entity through/apps/openregister/api/objects/decidesk/<schema>. Without it the app cannot provision its register at all and every spec fails against an empty UI. Now pinned toref: development(notmain), so the CI instance matches the environment the app is developed against.The register still would not provision itself.
occ app:enable decideskruns its repair step with no user session, OpenRegister's RBAC denies the import, and the step catches\Throwableand downgrades it to a warning —occ app:enableexits 0 with no register.Artefacts
tests/e2e/playwright.config.tschromium). The run step passes no--project, so whichever config it picks runs every project in it. The root config declaresdocs-capture(re-shoots documentation screenshots) andvisual(its own README records that a CI Linux runner cannot byte-match a dev-container PNG baseline).playwright-test-path: tests/e2emakes the workflow's first config lookup hit this file.testIgnoreis repeated at project level because a project-level list replaces the top-level one.tests/e2e/base-url.tsprocess.env.NEXTCLOUD_URL || 'http://localhost:8080'. AcceptsPLAYWRIGHT_BASE_URL | NEXTCLOUD_URL | NC_BASE_URL | BASE_URL—BASE_URLis the name the shared workflow actually exports — falls back to localhost:8080 only on CI, and throws otherwise. The old default was the shared dev container, and these specs seed real governance objects through the OR object API.tests/e2e/ci-seed.shPOST /apps/decidesk/api/settings/load, the only path that also merges theregister.d/fragments), then verifies the register slug + 18 schema slugs read verbatim out ofdecidesk_register.json, probes four object collections for readability, warms the SPA, and gates on the bundle serving as JavaScript (a missing bundle returns HTTP 200text/html, never 404).tests/e2e/.gitignorevisualproject's committed PNG baselines undertests/e2e/visual/*.spec.ts-snapshots/are not matched.What is deliberately NOT touched
The root
playwright.config.ts. The separateJourneydoc Capturejob greps it forname: 'docs-capture'and hard-fails without it. Thevisualanddocs-captureprojects are excluded by not declaring them in the new config, not by deleting them.Controls run locally
visual/ordocs-screenshotsentries.PLAYWRIGHT_BASE_URL/NEXTCLOUD_URL/NC_BASE_URL/BASE_URL/CIall cleared, the resolver raises rather than silently targeting the shared dev container.Pre-existing red
Integration Tests (Newman)andQuality Reportwere already failing ondevelopmentbefore this branch (run 30838121531, 29 jobs). They are not caused by this change.