Skip to content

[Visual Test] Label preflight rejects duplicate label, skipping Lavapipe screenshot capture #950

Description

@MichaelFisher1997

Workflow

https://github.com/OpenStaticFish/ZigCraft/actions/runs/30148576016

Failure output

label with name "run-visual-test" already exists; use `--force` to update its color and description
##[error]Process completed with exit code 1.

This is the full stderr emitted by the "Ensure visual-test label exists" step at 07:05:36.999Z in the failed run (build-output.log is never produced — the zig build run invocation is skipped because the workflow uses if: success() on the subsequent steps). The failure pattern is identical to runs 29677776782, 29725595085, 29809988559, 29899854913, 29987782599, 30074923197, and the visual-test workflow has been failing every day with this exact error since at least 2026-07-12.

Diagnosis

The visual-test workflow does not fail in the Vulkan / screenshot capture path. The Zig build and Lavapipe screenshot run never execute because the preflight gh label list / gh label create step exits non‑zero, which causes every later step (including the Run menu screenshot capture step that contains the actual zig build run -Dscreenshot-path=screenshot.png -Dskip-present=true invocation) to be skipped via if: success(). The "Failure output" above is therefore not a build-log message — build-output.log does not exist in the workspace because the build was skipped.

The actual root cause is in .github/workflows/visual-test.yml:55-68 ("Ensure visual-test label exists"):

if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then
  gh label create "visual-test" \
    --description "Issues from automated visual regression tests" \
    --color "E06C75"
fi
if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
  gh label create "run-visual-test" \
    --description "Run deterministic visual regression workflow on a PR" \
    --color "E06C75"
fi

gh label list paginates (default 30 per page) and --jq '.[].name'" only emits the **first page** of label names. The grep -q '^run-visual-test$'therefore returns 1 (no match) wheneverrun-visual-testis not on the first page, and the workflow then tries to recreate it. If the label *is* present (e.g., on a later page),gh label createerrors withlabel with name "run-visual-test" already exists; use --force to update its color and description. Either branch causes set -e` to abort the step with exit code 1.

The diagnostic prompt /.github/prompts/visual-test-diagnose.md misdirects the model toward a Vulkan / PPM failure that never runs — the "build-output.log" file referenced in that prompt is intentionally never created by the run-with-log action because the screenshot capture step is skipped before it executes.

Origin of the failure

  • File: .github/workflows/visual-test.yml
  • Step: Ensure visual-test label exists (lines 55-68)
  • Function: gh label list --json name --jq '.[].name' — paginated output, only first 30 labels returned
  • Logic: if ! ... grep -q '^run-visual-test$'; then gh label create ...; fi — recreates an already-existing label

Suggested fix

Either fetch all pages when listing labels, or skip create errors that already mean the label exists:

if ! gh label list --json name --jq '.[].name' --paginate | grep -q '^visual-test$'; then
  gh label create "visual-test" \
    --description "Issues from automated visual regression tests" \
    --color "E06C75" || true
fi
if ! gh label list --json name --jq '.[].name' --paginate | grep -q '^run-visual-test$'; then
  gh label create "run-visual-test" \
    --description "Run deterministic visual regression workflow on a PR" \
    --color "E06C75" || true
fi

Adding --paginate ensures run-visual-test is found on later pages, and || true on the create calls makes the step idempotent against transient gh label create failures (network/API errors, partial outages observed in earlier runner logs with Failed to restore: Cache service responded with 400 / Our services aren't available right now). The actual screenshot capture path (Lavapipe, headless swapchain at modules/engine-graphics/src/vulkan_swapchain.zig:127-174, PNG writer at modules/engine-graphics/src/vulkan/screenshot.zig:279-349, frame counting at src/game/app.zig:528-593) should then be exercised end-to-end and reported on its own merits.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghotfixstalevisual-testIssues from automated visual regression tests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions