Skip to content

ci: harden template build pipeline and expand rust test coverage - #440

Open
l50 wants to merge 6 commits into
dreadnode:mainfrom
l50:chore/sync-github-workflows
Open

ci: harden template build pipeline and expand rust test coverage#440
l50 wants to merge 6 commits into
dreadnode:mainfrom
l50:chore/sync-github-workflows

Conversation

@l50

@l50 l50 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Key Changes:

  • Introduced a reusable Docker Buildx composite action with BuildKit image pull retries to survive transient Docker Hub failures across all workflows
  • Added aggressive runner disk cleanup and cache tolerance so the large GPU template image builds no longer fail with "no space left on device"
  • Removed the entire Molecule test workflow and added a scheduled template build with automatic failure issue reporting
  • Expanded Rust unit test coverage for config resolution, host dedup logic, and loot snapshot normalization

Added:

  • Reusable Buildx setup action - Created .github/actions/setup-buildx/action.yml that retries BuildKit image pulls up to 5 times before failing, replacing scattered inline docker/setup-buildx-action usage
  • Scheduled template rebuilds - Added a weekly cron trigger and Rust source path filters (ares-cli, ares-core, ares-llm, ares-tools, Cargo.toml, Cargo.lock) to build-and-push-templates.yaml so template images rebuild when the baked ares binary changes
  • Automated failure reporting - Added a report-failure job that opens or updates a GitHub issue when template builds fail on main
  • Manual PR title validation - Added a workflow_dispatch path to semantic-prs.yaml for validating conventional-commit titles on demand
  • Renovate cargo-llvm-cov tracking - Added a regex manager to keep CARGO_LLVM_COV_VERSION in workflows up to date, pinned via new CARGO_LLVM_COV_VERSION env in rust.yaml
  • Proxmox placeholder env vars - Added Proxmox configuration placeholders to validate-templates.yaml
  • Extensive Rust tests - Added config path resolution, shipped-config validation, and set_model behavior tests in ares-cli/src/config.rs; host dedup tests in ops/loot/format/hosts.rs; and loot snapshot normalization tests in ops/loot/snapshot.rs

Changed:

  • Runner disk cleanup - Expanded the "Free up disk space" step to strip additional preinstalled toolchains, prune Docker images, and clean apt caches to reclaim ~25GB for GPU image builds
  • Buildx cache resilience - Added ignore-error=true to registry --cache-to exports so cache push failures don't fail builds
  • Renovate automerge strategy - Switched from branch-based to PR-based automerge and removed :automergeBranch preset
  • Pre-commit workflow - Added a retrying prefetch step for remote Taskfile includes and skip Rust hooks (cargo-fmt, cargo-clippy, cargo-check, cargo-test) since they run in the dedicated Rust workflow, trimming ~11 minutes
  • Release artifact handling - Made release.yaml fail loudly when an expected release binary is missing instead of silently skipping
  • Renovate fork processing - Enabled RENOVATE_FORK_PROCESSING so renovate processes forks
  • Clippy scope - Extended clippy to --all-targets in rust.yaml
  • Semgrep SARIF upload - Set continue-on-error: true so SARIF upload failures don't block the workflow
  • Template detection - Updated test-template-builds.yaml to scan warpgate-templates/templates/, pull in templates affected by ansible playbook changes, build without pushing, and reference the ghcr.io/l50/* registry
  • Action version bumps - Upgraded setup-python to v7.0.0, setup-go to v7, upload-artifact to v7.0.1, action-gh-release to v3.0.2, renovatebot/github-action to v46.2.1, and taiki-e/install-action across workflows

Removed:

  • Molecule test workflow - Deleted .github/workflows/molecule.yaml and its role change detection, matrix generation, and per-role test jobs

**Added:**

- Composite `setup-buildx` action that retries the BuildKit image pull with backoff to survive transient Docker Hub failures - `.github/actions/setup-buildx/action.yml`
- Weekly cron schedule and `ares-*` crate/Cargo path triggers to the template build workflow so images rebuild when the baked Rust binary sources change - `build-and-push-templates.yaml`
- `report-failure` job that opens or updates a tracking issue when template builds fail on main - `build-and-push-templates.yaml`
- Prefetch step for remote Taskfile includes with retries and a longer timeout to avoid transient CDN failures - `pre-commit.yaml`
- `workflow_dispatch` support with manual PR-number validation to the semantic PR title check - `semantic-prs.yaml`
- Proxmox placeholder env vars for template validation - `validate-templates.yaml`
- Ansible-aware change detection that maps changed playbooks and shared ansible files to affected templates - `test-template-builds.yaml`

**Changed:**

- Replaced inline `docker/setup-buildx-action` usages with the new composite action across build and test workflows
- Made buildcache export non-fatal via `ignore-error=true` and expanded the disk-cleanup step to reclaim ~25GB for GPU image builds - `build-and-push-templates.yaml`
- Bumped pinned actions: `checkout` v7.0.0→v7.0.1, `setup-python` v6.3.0→v7.0.0, `setup-go` v6→v7, `upload-artifact`, `action-gh-release`, `renovate`, `taiki-e/install-action`, and `dtolnay/rust-toolchain`
- Bumped `WARPGATE_VERSION` v4.9.1→v4.10.0 across template workflows
- Switched Renovate automerge from branch to PR type and enabled `RENOVATE_FORK_PROCESSING` - `renovate.json5`, `renovate.yaml`
- Skipped cargo hooks in pre-commit and ran clippy with `--all-targets` since Rust checks run in the dedicated Rust workflow - `pre-commit.yaml`, `rust.yaml`
- Made release binary packaging fail loudly when an expected binary is missing - `release.yaml`
- Made SARIF upload non-blocking with `continue-on-error` - `semgrep.yaml`
- Reworked template test builds to skip the push/pull round-trip and scope change detection to `warpgate-templates/templates/` - `test-template-builds.yaml`

**Removed:**

- Molecule test workflow - `.github/workflows/molecule.yaml`
@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.50%. Comparing base (c52f757) to head (111a682).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #440      +/-   ##
==========================================
- Coverage   81.33%   80.50%   -0.84%     
==========================================
  Files         476      460      -16     
  Lines      170395   161591    -8804     
==========================================
- Hits       138588   130084    -8504     
+ Misses      31807    31507     -300     
Files with missing lines Coverage Δ
ares-cli/src/config.rs 93.35% <100.00%> (+57.75%) ⬆️
ares-cli/src/ops/loot/format/hosts.rs 98.25% <100.00%> (+27.06%) ⬆️
ares-cli/src/ops/loot/snapshot.rs 78.73% <100.00%> (+78.73%) ⬆️

... and 17 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

l50 added 5 commits August 9, 2026 22:28
**Added:**

- Config command tests validating path resolution, shipped config parsing/rendering, and model-setting behavior including comment preservation, step budget retention, unknown-role rejection, and required-role enforcement - ares-cli/src/config.rs
- Host deduplication tests covering CIDR/empty-IP filtering, row merging, FQDN upgrades, DC stickiness, OS backfilling, role unioning, hostname-only folding, known-DC backfill, and IP sorting - ares-cli/src/ops/loot/format/hosts.rs
- Loot snapshot tests verifying domain normalization, key deduplication, identity case-folding, verbatim password/host/share handling, and collapse of repeated entries - ares-cli/src/ops/loot/snapshot.rs
- `CARGO_LLVM_COV_VERSION` environment variable to pin the cargo-llvm-cov tool version in the Rust workflow - .github/workflows/rust.yaml

**Changed:**

- Renovate configuration to track and update the pinned `cargo-llvm-cov` version via a regex manager for workflow files - .github/renovate.json5
- cargo-llvm-cov install step to use the pinned version from the environment variable - .github/workflows/rust.yaml
…orkflows

# Conflicts:
#	.github/workflows/molecule.yaml
@l50 l50 changed the title ci: harden template build workflows and refresh action pins ci: harden template build pipeline and expand rust test coverage Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/github Changes made to GitHub Actions workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant