fix(gates): gate-53 ignored the diff scope, gate-22 printed FAIL — 0, gate-33 had never run

.github/workflows/hydra-gates-package.yml

@@ -51,6 +51,35 @@ jobs:
       - name: Run the invariant suite
         run: bash hydra-gates/tests/test-hydra-gates-bin.sh
 
+      # FIRST, with no node_modules at all — this is the only state in which the
+      # DEGRADED contract can be exercised. Gate 22's whole point is that a
+      # validator which could not resolve Ajv must exit 3 and say so, never 0;
+      # once ajv is installed below, `NODE_PATH=/nonexistent` no longer hides it
+      # (node still resolves from ./node_modules) and that assertion self-skips.
+      - name: "Manifest validator: the no-Ajv degraded contract"
+        run: bash hydra-gates/scripts/lib/test_check_manifest.sh
+
+      # Ajv is what gates 22 and 53 validate manifests WITH. Without it the
+      # vendored validator can only run its structural lint, and the suite below
+      # would skip exactly the paths worth testing. It is a devDependency of
+      # every fleet app already; installing it here means CI exercises the same
+      # code path a real `npm ci` repo does.
+      - name: Install ajv for the manifest-validation helpers
+        run: npm install --no-save --no-audit --no-fund ajv ajv-formats
+
+      # These three cover the gate-22 verdict contract and the gate-53 ADR-020
+      # diff scoping. They lived in scripts/lib/ and were run by NOTHING —
+      # test_check_manifest.sh in particular had been green for its whole life
+      # while pointing at a fixture directory that did not exist (a missing
+      # manifest path makes the validator print "Tier 0, skipping" and exit 0,
+      # which is what two of its three assertions expected).
+      - name: Manifest gate helper suites
+        run: |
+          set -eu
+          bash hydra-gates/scripts/lib/test_check_manifest.sh
+          node hydra-gates/scripts/lib/test_manifest_scope_filter.js
+          python3 hydra-gates/scripts/lib/test_manifest_diff_scope.py
+
   install-from-published-location:
     name: "Install from published location (php:8.3-cli)"
     runs-on: ubuntu-latest

.github/workflows/quality.yml

@@ -2698,9 +2698,14 @@ jobs:
         # Installed INTO the package dir so `require('ajv')` from
         # scripts/lib/check_manifest.js resolves by walking up to
         # gates/hydra-gates/node_modules. A global install would need NODE_PATH
-        # and would silently not be found — which the manifest gates survive via
-        # a vendored fallback, meaning the miss would not be visible here. The
-        # coverage block at the end of the run is what makes any such gap loud.
+        # and would silently not be found.
+        #
+        # Since 2026-08-03 a miss here is loud rather than survivable: gate-22
+        # used to fall back to a structural lint that checks only the AppHost
+        # blocks and report the result as a PASS — a manifest certified without
+        # ever being schema-validated. It now FAILS with a named reason (the
+        # validator exits 3 DEGRADED), exactly as gate-53 already refused to run
+        # fail-open. If this step is ever removed, both gates say so.
         run: npm --prefix gates/hydra-gates install --no-save --no-audit --no-fund ajv
 
       - name: Resolve the diff base

hydra-gates/README.md

@@ -73,14 +73,45 @@ Then `composer update conduction/hydra-gates`.
 `vendor/conduction/hydra-gates` lands at about 1.2 MB. The org profile, the
 website and the docs tree are `export-ignore`d and do not follow.
 
+### Upgrading to `v1.1.0` from `v1.0.x`
+
+`^1.0` picks this up on the next `composer update`, and **verdicts move**. Three
+things to do before you upgrade:
+
+1. **Make `ajv` resolvable before the gates run** — `npm ci`, or
+   `npm --prefix <package-dir> install --no-save ajv` (Option A's shared
+   workflow already does this). Gate 22 previously fell back to a structural
+   lint that checks only the AppHost blocks and reported the result as a PASS;
+   it now **fails** with a named reason rather than certifying a manifest it
+   never schema-validated. Gate 53 has always refused to run without it.
+2. **Expect gate-22 verdicts to move in both directions.** Its verdict used to
+   come from the app's own `npm run check:manifest`; it now comes from the
+   vendored canonical validator, and the app script is surfaced as an advisory.
+   Apps whose local checker was weaker will surface real findings; apps failed
+   by a stale app-local page-type enum will go green.
+3. **Treat `: SKIPPED` as "did not run", not as a pass**, if you parse
+   `^\[gate-N\]` lines. New verdict; see *Reading a green* below.
+
+The upside: **gate 53 becomes usable under `--scope-to-diff`**. It was
+previously unenablable on any repo with manifest debt, because a one-line change
+reproduced the full-repo finding count exactly.
+
 ---
 
 ## What it needs at runtime
 
 `bash`, `git`, `python3` (about twenty gates are Python helpers) and `node`
-(gates 22 and 53 only, and they additionally want `ajv` resolvable). PHP is
-required only because composer is one of the two delivery mechanisms; **no gate
-executes PHP**.
+(gates 22 and 53 only). PHP is required only because composer is one of the two
+delivery mechanisms; **no gate executes PHP**.
+
+**Gates 22 and 53 need `ajv` resolvable, and both now say so instead of
+degrading quietly.** `ajv` is already a transitive devDependency in every fleet
+app's `package-lock.json`, so a `npm ci` resolves it; a bare checkout without
+`node_modules` does not. Gate 53 has always refused to run without it. Gate 22
+used to fall back to a structural lint that checks only the AppHost blocks and
+report the result as a PASS — a manifest certified without ever being
+schema-validated. It now fails with a named reason instead. Set `NODE_PATH` or
+run `npm ci` before the gates if you see it.
 
 **No gate needs a Nextcloud runtime.** Nothing under `scripts/` loads
 `../../lib/base.php` — that constraint belongs to `phpunit`, not to the gates,
@@ -147,25 +178,95 @@ clean one. So:
   construction.
 - A genuinely empty diff is **stated as empty** rather than reported as a pass.
 
+### Scope granularity, and where file granularity is not enough
+
+Most gates scope by **file**: a finding in a file the PR did not touch does not
+block. Gates 51 (schema-property-titles) and 55 (detail-page-discipline) go
+finer and scope by the **changed lines** inside a touched file, so legacy debt
+elsewhere in a file you edited does not block either.
+
+Gate 53 (effective-manifest-crossref) used to scope by file, and for that gate
+file granularity was indistinguishable from no scoping at all: an app's entire
+navigation surface lives in `src/manifest.json` + `src/manifest.d/*`, so
+touching any of it re-judged all of it. Measured 2026-08-03 on a one-line
+`title` change:
+
+| repo | full-repo | diff-scoped (before) | diff-scoped (after) |
+| --- | --- | --- | --- |
+| pipelinq | 24 | 24 | 0 blocking, 24 reported PRE-EXISTING |
+| shillinq | 246 | 246 | 0 blocking, 397 reported PRE-EXISTING |
+
+Gate 53 now separates two things that are not the same:
+
+- **Answering** a cross-reference needs the whole assembled manifest. You cannot
+  resolve `menu[].route` → page id, or check the ADR-044 no-orphan-removal
+  invariant, from a diff. That part of the gate is legitimately whole-repo and
+  stays so.
+- **Blocking** on the answer does not. Every finding carries a JSON pointer that
+  resolves to a page id, a menu id or a top-level block, and it blocks only when
+  the PR touched that entry.
+
+Findings on untouched entries are **printed as `PRE-EXISTING`, never dropped**,
+and their count is reported on stdout, so a scoped green cannot be read as "the
+manifest is clean". Findings that address the manifest as a whole (no entry to
+attribute them to), and any PR whose scope cannot be determined — a brand-new
+fragment untracked at base, a changed register JSON, a parse failure — block
+regardless. Unverifiable scope is never treated as narrow scope.
+
+The only part of the suite that is whole-repo **by nature** — as opposed to by
+oversight — is that residual set of gate-53 invariants. Everything else measured
+on 2026-08-03 (gates 34, 51, 55) narrows correctly, and the two other
+whole-manifest checks in the same family, gates 22 and 52, are triggered only
+by a change to the artefact they judge.
+
 ---
 
 ## Reading a green
 
 A green from this package says how much it covers, because a green that
 overstates its coverage is the same defect as `|| echo '...skipping'` one layer
-up. The runner's own closing line reads `ALL 61 GATES GREEN` regardless of how
-many gates ran; measured on openbuild, 59 of 61 report and gates 24 and 33 skip
-silently when their prerequisites are absent. So every run ends with:
+up.
+
+Every gate in the runner is wrapped in a prerequisite test (`if [ -d src ]`,
+`if [ -f tests/axe/report.json ]`, …). Until 2026-08-03 a gate whose
+prerequisite was absent emitted **nothing at all** — no line, no count, no
+trace — and the runner still closed with `ALL 63 GATES GREEN`. Measured across
+13 fleet repos, **gate 33 (axe-core) had never run in any of them**: the
+`tests/axe/report.json` it consumes is produced by a `scripts/run-browser-tests.sh`
+that exists in no app, while `axe-core` sits in every app's `devDependencies`
+so the prerequisite looks wired. Every green the fleet had ever produced
+excluded accessibility runtime checking, and nothing said so. Gate 24
+(integration-parity) was absent in most repos for the same structural reason.
+
+Both layers now account for it. A gate that cannot run says so on its own line:
+
+```
+[gate-24] integration-parity: SKIPPED — no scripts/check-integration-parity.sh …
+[gate-33] axe-core: SKIPPED — no tests/axe/report.json in this repo — axe-core
+          never ran against a rendered DOM, so contrast / landmark /
+          ARIA-validity / live-region accessibility is UNVERIFIED. …
+```
+
+and every run — `bin/hydra-gates` **and** a direct `run-hydra-gates.sh`
+invocation — ends with the accounting:
 
 ```
-[hydra-gates] COVERAGE: 59 of 61 declared gates reported a result.
-[hydra-gates] GATES THAT DID NOT RUN: 24 33
-[hydra-gates] RESULT: ALL GATES PASSED — EXCEPT GATES 24 33, WHICH DID NOT RUN.
-[hydra-gates] This green covers 59 gates. It says NOTHING about gates 24 33.
+[hydra-gates] COVERAGE: 60 of 63 declared gates reported a result.
+[hydra-gates] GATES THAT DID NOT RUN — they inspected NOTHING, and their subject
+[hydra-gates] matter is UNVERIFIED by this run:
+[hydra-gates]   gate-4 composer-audit
+[hydra-gates]   gate-24 integration-parity
+[hydra-gates]   gate-33 axe-core
+[hydra-gates] 60 GATE(S) GREEN — but 3 of 63 DID NOT RUN (named above).
+[hydra-gates] This is NOT 'all 63 gates green'. …
 ```
 
+A `SKIPPED` line is **not** counted as coverage — a gate reporting that it did
+nothing did nothing. `--require-full-coverage` turns an incomplete run into
+exit 98.
+
 The inventory is read out of the runner itself rather than hardcoded, so adding
-gate 62 does not silently leave the coverage check measuring against a stale 61.
+gate 64 does not silently leave the coverage check measuring against a stale 63.
 
 ### Waivers
 
@@ -188,14 +289,29 @@ A green earned by passing is then distinguishable from one earned by waiving.
 
 ```
 bash hydra-gates/tests/test-hydra-gates-bin.sh     # or: composer test:package
+
+# manifest gate helpers (gate 22's verdict contract, gate 53's diff scoping).
+# Install ajv first or the schema paths state a SKIP instead of passing:
+npm install --no-save ajv ajv-formats
+bash hydra-gates/scripts/lib/test_check_manifest.sh
+node hydra-gates/scripts/lib/test_manifest_scope_filter.js
+python3 hydra-gates/scripts/lib/test_manifest_diff_scope.py
 ```
 
-Asserts the four invariants — exit-code-is-count, loud unresolvable base, stated
-empty diff, self-describing coverage — against a synthesized fixture repo. The
-positive control runs in **both** directions: the injected violation must be
-*named* by the gate that catches it, and the same fixture must go green once it
-is removed. A one-directional control cannot distinguish "the check caught it"
-from "the check never ran".
+Asserts the invariants — exit-code-is-count, loud unresolvable base, stated
+empty diff, self-describing coverage, named unrun gates, one-line-per-verdict —
+against a synthesized fixture repo. The positive control runs in **both**
+directions: the injected violation must be *named* by the gate that catches it,
+and the same fixture must go green once it is removed. A one-directional control
+cannot distinguish "the check caught it" from "the check never ran".
+
+`test_check_manifest.sh` was itself an example of that failure until
+2026-08-03: it pointed at a `scripts/test-fixtures/manifest-validation/`
+directory that had never existed, and `check_manifest.js <missing-path>` prints
+"Tier 0, skipping" and exits 0 — which is exactly what two of its three
+assertions expected. It had been green its whole life while validating nothing.
+The fixtures now ship, and the suite refuses to run at all if they go missing
+again.
 
 CI for this package lives in
 [`.github/workflows/hydra-gates-package.yml`](../.github/workflows/hydra-gates-package.yml):

hydra-gates/bin/hydra-gates

@@ -28,13 +28,21 @@
 #      shallow clone it may not exist at all.
 #
 #   3. A COVERAGE ASSERTION ON THE GREEN.
-#      The runner ends with "ALL 61 GATES GREEN" whether or not 61 gates ran.
-#      Measured on openbuild, only 59 of the 61 emit a line — gates 24 and 33
-#      skip silently when their prerequisites are absent. A green that counts
-#      gates it never executed is the same defect as `|| echo 'skipping'`, one
-#      layer up. We diff the gate numbers that actually reported against the
-#      inventory declared by the runner itself, and say plainly which gates did
-#      not run and what the green therefore does not cover.
+#      The runner used to end with "ALL 63 GATES GREEN" whether or not 63 gates
+#      ran. Measured on openbuild, only 59 of 61 emitted a line — gates 24 and
+#      33 skipped silently when their prerequisites were absent; measured
+#      2026-08-03 across 13 repos, gate-33 (axe-core) has never run in ANY of
+#      them. A green that counts gates it never executed is the same defect as
+#      `|| echo 'skipping'`, one layer up. We diff the gate numbers that
+#      actually reported against the inventory declared by the runner itself,
+#      and say plainly which gates did not run and what the green therefore
+#      does not cover.
+#
+#      As of 2026-08-03 the runner performs the same accounting in its own
+#      summary, so a direct `run-hydra-gates.sh` invocation (which most of the
+#      fleet uses) is no longer blind to it. Gates that state their own absence
+#      now print `[gate-N] <name>: SKIPPED — <reason>`; that line is NOT counted
+#      as "reported" below — a gate that says it did nothing did nothing.
 #
 #   4. A LOUD, STATED SKIP FOR MISSING PREREQUISITES.
 #      Never `|| echo '...skipping'`. A prerequisite that is genuinely absent is
@@ -267,8 +275,13 @@ RC="${PIPESTATUS[0]}"
 # adds gate 62, and a coverage check that silently measures against a stale
 # inventory is the very defect this step exists to catch.
 # ---------------------------------------------------------------------------
-DECLARED="$(grep -oE '_(pass|fail) [0-9]+' "${RUNNER}" | awk '{print $2}' | sort -un)"
-EMITTED="$(grep -oE '^\[gate-[0-9]+\]' "${_out}" | grep -oE '[0-9]+' | sort -un)"
+DECLARED="$(grep -oE '_(pass|fail|skip) [0-9]+' "${RUNNER}" | awk '{print $2}' | sort -un)"
+# A `[gate-N] name: SKIPPED — reason` line is the gate REPORTING THAT IT DID NOT
+# RUN. Counting it as coverage would turn the fix into the bug: the whole point
+# of making the skip visible is that it stays outside the "reported a result"
+# tally. Excluded here by verdict, not by gate number.
+EMITTED="$(grep -E '^\[gate-[0-9]+\]' "${_out}" | grep -v ': SKIPPED' \
+    | grep -oE '^\[gate-[0-9]+\]' | grep -oE '[0-9]+' | sort -un)"
 DECLARED_N="$(printf '%s\n' "${DECLARED}" | grep -c . || true)"
 EMITTED_N="$(printf '%s\n' "${EMITTED}" | grep -c . || true)"