Skip to content

test(cli): fix the two remaining Full Suite lane failures - #37

Open
timoteo7 wants to merge 1 commit into
mainfrom
fix/ci-screenshot-reference-drift
Open

timoteo7 wants to merge 1 commit into
mainfrom
fix/ci-screenshot-reference-drift

Conversation

@timoteo7

Copy link
Copy Markdown
Owner

Continuação do #36

Depois do merge do #36 o shard 3 do Full Suite ainda falhava com exatamente 2 arquivos em packages/cli (169 passed | 2 failed | 3 skipped). Ambas as falhas eram drift pré-existente que o #36 não causou nem corrigiu.

1. task-retry.test.ts — o produto estava certo, o teste estava errado

AssertionError: expected 'in-review' to be 'todo'

O fixture é uma falha de merge (todos os steps done, mergeRetries: 4). O commit 706c15560 (FN-9317, "recover stalled in-review merges") deliberadamente mudou essa forma para retry no lugar: limpa status/error/auto-pause, zera mergeRetries e mantém o card em review para não reexecutar trabalho já aprovado. A expectativa todo é drift anterior ao FN-9317, vindo de 1c69ea78b (FN-6173), que enviava retries de volta para todo e depois foi revertido.

Todas as outras superfícies já afirmavam o comportamento no lugar para essa mesma forma:

  • commands/__tests__/task.test.ts — moveTask NÃO é chamado, e registra "in-review merge retry, mergeRetries reset"
  • __tests__/extension.test.ts — details.newColumn === "in-review"
  • dashboard register-task-workflow-routes.ts — classifica igual

O teste do CLI era o único atrasado. Corrigi a expectativa, renomeei o teste para declarar o contrato que ele fixa, e adicionei o caso complementar (falha de execução, steps incompletos -> re-enfileira no hold preservando progresso), espelhando extension.test.ts. Cobertura aumentou: 3 passed / 1 failed -> 4 passed.

2. skills-get.test.ts — não era hang, era formato

Test timed out in 5000ms

O teste fazia três boots seriais do bundle ESM de ~19 MB (dist/bin.js) dentro de um único it: guide, --version e o caminho de erro. Cada boot custa ~1,0 s local e 2-3 s num runner compartilhado. Três boots seriais = ~3,1 s local contra o default de 5000 ms do Vitest, e estoura no CI. Nada travava.

Correção na costura, não no orçamento: os dois spawns independentes (guide e --version) rodam concorrentes via Promise.all, e o boot do caminho de erro vai para o seu próprio it. Tempo cai de ~3,1 s para ~1,2 s com as mesmas asserções e sem aumentar timeout — check-no-test-timeout-appeasement.mjs continua limpo. 6/6 passam.

Verificação

  • task-retry 4/4, skills-get 6/6, runTaskRetry 12/12
  • tsc --noEmit limpo, check-no-test-timeout-appeasement.mjs limpo, check-changeset-format.mjs limpo, eslint 0 erros
  • Sem changeset: mudança só de teste, sem delta de comportamento publicado

O que NÃO está neste PR

Pipeline smoke tier continua vermelho, e é outra coisa: orçamento, não asserção. Medido localmente, o projeto engine-pipeline-smoke leva 160,1 s contra PIPELINE_SMOKE_DURATION_BUDGET_MS = 175_000 — só ~9% de folga, e o wrapper orça a lane inteira numa única invocação. O arquivo mais lento sozinho (pipeline-resilience.pipeline.test.ts, 157,0 s) quase consome o orçamento inteiro dos 3 workers. Num runner do GitHub isso estoura o watchdog. O comentário do código diz que os 175 s são "a regression detector, not a knob for hiding overruns", então subir o orçamento é exatamente o remendo que a política do repo proíbe. A correção legítima é reduzir o custo por cenário ou aumentar workers/shardar a lane.

…rence-drift

Two packages/cli tests failed in the Full Suite shard on b4cddcb. Both were
pre-existing drift that PR #36 (test files only) neither caused nor fixed.

1. task-retry.test.ts "clears the deadlock auto-pause" — expected 'todo', got
   'in-review'. The fixture is a MERGE failure (all steps done, mergeRetries 4),
   and FN-9317 ("recover stalled in-review merges", 706c155) made that shape
   retry IN PLACE: clear status/error/auto-pause, reset mergeRetries, keep the
   card in review so approved work is not re-run. The `todo` assertion is
   pre-FN-9317 drift from FN-6173 (1c69ea7), which briefly sent CLI merge
   retries back to todo and was later reverted. Every sibling surface already
   asserts the in-place behavior for this exact shape: commands/__tests__/task.test.ts
   asserts moveTask is NOT called and logs "in-review merge retry, mergeRetries
   reset"; __tests__/extension.test.ts asserts details.newColumn === "in-review";
   the dashboard route classifies it the same way. The product is correct, so the
   stale expectation is fixed and the test is renamed to state the contract it
   pins. Added the complementary execution-failure case (unfinished steps -> re-queue
   to the hold column with progress preserved), mirroring extension.test.ts, so the
   re-queue half of the deadlock auto-pause stays covered. 3 passed / 1 failed -> 4 passed.

2. skills-get.test.ts "prints a guide and version from the same built CLI entry
   point" — 'Test timed out in 5000ms'. The test did three SERIAL cold boots of
   the ~19 MB dist/bin.js ESM bundle inside one `it` (~1.0 s each locally, 2-3 s on
   a shared GitHub runner): the guide, --version, and the unknown-skill error path.
   Three serial boots need ~3.1 s locally against Vitest's 5000 ms default and
   exceed it on the runner. Nothing hung; the seam is what was wrong. Fix: run the
   two independent guide/version invocations concurrently via Promise.all so their
   boots overlap, and move the error-path boot into its own test. Wall clock drops
   from ~3.1 s to ~1.2 s (measured) with identical assertions and no timeout bump
   (check-no-test-timeout-appeasement.mjs stays green). Coverage unchanged: the
   guide must still come from the built entry point and its embedded version must
   still match that same built binary's --version.

Gates: task-retry 4/4 pass, skills-get 6/6 pass, runTaskRetry suite 12/12 pass,
cli tsc --noEmit clean, check-no-test-timeout-appeasement.mjs clean,
check-changeset-format.mjs clean, eslint 0 errors (test files are eslint-ignored).
No changeset: test-only change, no published behavior delta (AGENTS.md).

Pipeline smoke tier is a SEPARATE pre-existing timing failure, not touched here.
@github-actions

Copy link
Copy Markdown

ThreatCrush Security Scan

4589 finding(s)

HIGH/CRITICAL: 43 | MEDIUM: 4035 | LOW: 511

Severity Rule Location
HIGH secret-database-url .github/workflows/full-suite.yml:55
HIGH secret-generic-credential .github/workflows/full-suite.yml:56
HIGH secret-database-url .github/workflows/full-suite.yml:284
HIGH secret-generic-credential .github/workflows/full-suite.yml:285
HIGH secret-database-url .github/workflows/full-suite.yml:324
HIGH secret-generic-credential .github/workflows/full-suite.yml:325
HIGH secret-database-url .github/workflows/pr-checks.yml:221
HIGH secret-generic-credential .github/workflows/pr-checks.yml:222
HIGH secret-generic-credential .github/workflows/release.yml:522
HIGH secret-generic-credential .github/workflows/release.yml:524
HIGH secret-generic-credential .github/workflows/test-release.yml:445
HIGH secret-generic-credential .github/workflows/test-release.yml:447
HIGH secret-generic-credential docs/cli-reference.md:80
HIGH secret-generic-credential docs/signals-connectors.md:34
HIGH secret-generic-credential docs/signals-connectors.md:77
HIGH secret-generic-credential docs/signals-connectors.md:94
HIGH secret-generic-credential docs/signals-connectors.md:117
HIGH secret-generic-credential docs/signals-connectors.md:159
HIGH secret-generic-credential packages/cli/STANDALONE.md:71
HIGH secret-database-url packages/core/src/postgres/credential-redact.ts:12
HIGH secret-database-url packages/core/src/postgres/credential-redact.ts:30
HIGH secret-database-url packages/core/src/postgres/credential-redact.ts:31
HIGH secret-database-url packages/core/src/postgres/credential-redact.ts:102
HIGH secret-database-url packages/core/src/postgres/credential-redact.ts:103
HIGH secret-generic-credential packages/core/src/postgres/embedded-lifecycle.ts:843
HIGH secret-database-url packages/core/src/postgres/embedded-lifecycle.ts:1574
HIGH secret-database-url packages/core/src/postgres/pg-backup.ts:756
HIGH js-ssrf-outbound-request packages/dashboard/app/public/sw.js:651
HIGH js-ssrf-outbound-request packages/dashboard/app/public/sw.js:727
HIGH js-host-header-trust packages/dashboard/src/cli-session-ws.ts:81
HIGH js-host-header-trust packages/dashboard/src/cli-session-ws.ts:115
HIGH js-ssrf-outbound-request packages/dashboard/src/routes.ts:1858
HIGH js-host-header-trust packages/dashboard/src/server.ts:2689
HIGH js-host-header-trust packages/dashboard/src/server.ts:2714
HIGH js-host-header-trust packages/dashboard/src/server.ts:3025
HIGH js-host-header-trust packages/dashboard/src/server.ts:3193
HIGH secret-slack-webhook plugins/examples/fusion-plugin-notification/README.md:46
HIGH secret-database-url scripts/pg-test-server.mjs:200
HIGH secret-database-url scripts/pg-test-server.mjs:231
HIGH secret-database-url scripts/pg-test-server.mjs:241
HIGH secret-generic-credential scripts/sync-fusion-skill-tools.mjs:550
HIGH secret-generic-credential scripts/verify-windows-elevated-restricted.mjs:81
HIGH secret-generic-credential scripts/verify-windows-encoding-recovery.mjs:41
MEDIUM redos-nested-quantifier docs/agents.md:1710
MEDIUM insecure-temp-file docs/solutions/test-failures/main-full-suite-census-2026-09-25.md:376
MEDIUM insecure-temp-file docs/solutions/test-failures/main-full-suite-census-2026-09-25.md:377
MEDIUM insecure-temp-file docs/solutions/test-failures/main-full-suite-census-2026-09-25.md:541
MEDIUM insecure-temp-file packages/cli/src/__tests__/bin.test.ts:136
MEDIUM insecure-temp-file packages/cli/src/__tests__/dev-with-memory-lib.test.ts:33
MEDIUM insecure-temp-file packages/cli/src/__tests__/dev-with-memory-lib.test.ts:34

…and 4539 more. Full results in the Security tab.

Snippets are redacted; ThreatCrush never prints matched credential material.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant