Development

Development notes

CI behavior when working on agent-box itself. Repo conventions — structure, build and test commands, commit style — live in AGENTS.md at the repo root, so this page does not repeat them.

Host-specific setup is deliberately absent. How a given box puts tools on PATH, sets a git identity or stores a gh token is a property of that box, not of the template, so it belongs in that host's own config repo (for the maintainer's box, a private one).

CI

• Watch runs in the FOREGROUND: gh run watch <id> --exit-status. Don't background a streaming watch — it leaves stale "Running" ghost tasks in the UI.
• CI only runs on build-relevant paths (including Nix, payloads, tests, scripts, the lockfile and the CI workflow); doc-only pushes skip it.
• The landing page (docs/) publishes via GitHub Pages branch source, not a workflow — a push to master that touches docs/ deploys on its own.
• modules/agent-box.nix is generated. After you edit modules/agent-box.nix.in or modules/src/, run nix run .#assemble and commit the result, or the module-generated-up-to-date check fails on drift.

Validation scheduling and timing (#519)

Implemented in PR #671, which records baseline measurements, validation runs and the compute tradeoff.

CI runs native checks alongside four VM lanes on standard runners. Each VM lane prepares only its own drivers before executing one test at a time, so at most four VM tests run per workflow run. A test can start multiple guest machines. The extra runners reduce the critical path at the cost of more total runner minutes; do not compare only the fastest job or silently raise VM concurrency.

Validate module & VM is the final gate. It runs even when a dependency fails and succeeds only if native validation and every VM lane succeeded. The matrix keeps running sibling lanes after a failure so their results are not lost. For performance measurements, count from the first validation job's start through the final gate's completion, not the short gate alone. Record initial queue time separately, per-lane preparation and VM execution times, and the sum of job durations to expose the compute tradeoff.

Native check discovery and VM lane inventory checks replace the old manual workflow check list. Add native checks to the flake; register VM tests in vmTestsFor and assign each exactly once in ciVmLanes. The regression check also compares that schedule with the workflow matrix and checks the final gate's handling of failures, cancellations and skipped jobs.

Moving VM tests onto less busy runners exposed races in fixtures that injected state while a real CLI was still starting. The rotation fixture's synthetic SessionStart event could be overwritten by the real startup hook; the notice fixture's synthetic transcript could change during a resume. The quiesce_harness helper in tests/sessions.nix replaces the pane with an idle process and waits for the old process group to disappear before supplying synthetic state. Keeping the session present prevents the supervisor from respawning it early. The test then kills the pane and checks the real supervisor's next command, preserving both rotation hops and all notice cases.