feat(agents): durable managed-agent runtime (configurable orchestration workflows)

[jlwainwright]

Closes #3871
References #5523 (docs-only spec for reliable job lifecycle — this PR supplies the working implementation of both proposals)

What

Adds a durable managed-agent runtime to the ACP harness: jobs survive runner crashes and runtime restarts, with bounded state, enforced capability separation, and a Nostr-first job event surface.

New crate: buzz-runtime

• Detached job supervisor with process-group fencing (POSIX process groups, Windows named Job Objects with owner-only SDDL)
• Loopback-only control server with constant-time token auth; non-loopback bind refused
• Capability model: the Model role cannot JobsCancel/Reconcile/Shutdown
• Artifacts with runner receipts (schema-versioned, identity + argv-digest validated), bounded logs, bounded inbox
• All bounds (MAX_*) enforced at protocol, supervisor, and store layers

Relay surface (Nostr-first)

• New event kinds 43001–43006 pre-registered in buzz-core/src/agent_job.rs with doc comments
• GET /jobs + GET /jobs/{job_id} indexed-reads endpoints (NIP-98 auth; job writes go through the existing POST /events) — the HTTP exception AGENTS.md reserves for genuinely HTTP-only surfaces
• Ingest validates channel tag for job kinds (no expect on this path)
• Migration 0028_agent_jobs.sql (idempotent, count test updated)

Desktop

• Managed-agent job projection with strict artifact URI allowlist (https?:// | nostr:), strict sha256 validation, no arbitrary key acceptance
• types.ts kept under the 1000-line file-size cap

CLI

• buzz jobs stop signs KIND_JOB_CANCEL via buzz_sdk (Nostr-first, no new HTTP)

Security

A full security audit of the new surfaces found no critical or high findings. Cheap fixes already applied in this PR:

• Artifact URI scheme allowlist in the desktop projection (SEC-001)
• Channel-tag guard in relay job ingest (SEC-004)

Enforced surfaces verified: loopback-only control server with constant-time token; non-loopback control_addr refused; Model capability cannot cancel/reconcile/shutdown; cwd confinement; argv_sha256 cross-check (no shell invocation — argv array passed directly); env scrubbing; relay state machine row-locked; bounds enforced at all three layers.

The crate keeps #![deny(unsafe_code)]; the two exempted FFI sites (macOS proc_listpids, Windows Job Objects) carry documented safety invariants (// SAFETY: comments / module-level safety doc).

Conventions

• DCO: all commits signed off
• All 118 new pub items in buzz-runtime have rustdoc comments
• No new unwrap()/expect() in production paths (three known crate allowances unchanged)
• Migration counted in embedded-migrator test
• Desktop file-size ratchet preserved

Testing

• Full unit suite: 910 passed
• Full integration suite (Postgres + Redis via docker compose): 1214 passed
• just check green (fmt, clippy root+tauri with -D warnings, biome desktop/web, mobile analyze, file-size caps, px-text guard, pubkey-truncation guard)

Companion spec: #5523. Related prior art discussions: #4161 (FileFailurePersistence), #4638 (buzz-dev-mcp ZeroClaw ACP), #2426 (JOB_DECLINED).

Post-draft quality pass

• Coverage boost: buzz-runtime line coverage 59% -> 67%. New tests: live-server RuntimeClient round-trips (controller hello/status/reconcile/shutdown, model-capability denial, dead-pid receipt rejection) and artifact contract tests (spec round-trip + tamper rejection, runner-receipt terminal invariants, attempt-dir normalization, workspace containment).
• Coverage map (remaining gaps, by risk): store.rs 36% of file — recovery scans, inbox requeue, outbox drain paths are exercised only indirectly by integration suites; protocol.rs 40% — wire-struct Debug/From impls (test-only cost); artifacts.rs 47% — canonicalize edge paths. Highest-risk uncovered = crash-recovery requeue loop; covered indirectly by buzz-acp job_supervisor (69%) + relay agent_jobs handler (91%) suites.
• Public API docs: crate now carries #![warn(missing_docs)]; all 372 flagged items documented.
• Canonical job states: eight-state lifecycle single-sourced as buzz_core::agent_job::AGENT_JOB_STATES; relay + CLI --state filters consume it (drops two hand-maintained copies that could drift from the 0028_agent_jobs.sql CHECK).
• Four confirmed small bugs fixed pre-draft (reconcile contract, sticky cancelling, unbounded log tail, unbounded constants) in commit 53d39fa.

[themiguelamador]

Requesting changes after a full pass over the durable job protocol, relay projection, runtime control/store/recovery paths, ACP integration, CLI, SDK, and migration.

I pushed the fixes that were safe to make without redesigning the runtime as commit 82a7bdfa2. That commit:

• renumbers the colliding 0028_agent_jobs.sql migration to 0051 (0028–0050 are already allocated across this repository's branches);
• makes target-side pre-accept rejection durable and publishable, and lets the relay accept the corresponding attempt-1 terminal event;
• tightens request/result/error/artifact validation, bounds artifact reads and control errors, and aligns CLI/SDK/desktop URI behavior;
• bounds authenticated control connections, pending inbox admission, local job listings, and capacity rejection storage;
• preserves relay write acknowledgements in CLI output and removes new production expect paths.

Two merge blockers remain:

1. The runtime adds hand-written unsafe outside the sole allowed generated mobile FFI exception. crates/buzz-runtime/src/artifacts.rs, crates/buzz-runtime/src/windows_job.rs, crates/buzz-acp/src/job_runner.rs, and crates/buzz-acp/src/job_windows.rs call platform APIs through many unsafe blocks (some with local allow(unsafe_code) overrides). This directly conflicts with the repository-wide policy. Please replace these paths with safe abstractions, or obtain and document the required project-level security decision rather than locally bypassing the lint.

2. The durable runtime has no retention/compaction policy. Completed/dead-letter inbox rows, terminal jobs/assignments, and published/rejected/superseded outbox rows are never removed. The active-admission caps therefore do not bound total disk growth. In addition, cancel-before-request tombstones have a global 4,096-row cap but no pruning, so a long-lived runtime eventually rejects every new tombstone permanently. Add a transactional retention/compaction policy with tests proving active work, replay/idempotency fences, and necessary terminal history are preserved.

Minor documentation issue: the PR description says the relay adds POST /jobs; the implementation adds GET /jobs and GET /jobs/{id}, while writes continue through POST /events. Please correct the API summary.

Verification completed on the fix commit:

• affected-package cargo clippy --all-targets -- -D warnings: pass;
• buzz-runtime library: 29 passed;
• ACP job-supervisor tests: 15 passed;
• buzz-dev-mcp library: 123 passed, 1 ignored;
• CLI jobs tests: 5 passed;
• SDK agent-job tests: 8 passed;
• database migration lint: 7 passed, 3 Postgres-only tests ignored.

The focused relay test binary could not finish codegen because the host volume reached ENOSPC; this was an environment failure, not a test assertion. The relay code did pass the affected-package clippy build. The new Postgres regression remains infrastructure-gated and was not run because no local database container was available.

[jlwainwright]

Both review blockers addressed in 18f5c8f:

1. Retention/compaction — added a transactional compact() on the durable store: terminal inbox rows, terminal jobs (excluding any linked to a still-active assignment), terminal assignments, and settled outbox rows are removed past 7-day cutoffs; cancel tombstones for jobs with no live state are pruned back under the 4,096 cap, so a long-lived runtime no longer permanently rejects new tombstones. It runs automatically once per store-thread hour and compact_at() is exposed for tests. store::tests::compaction_preserves_active_work_and_fences proves active work, replay fences and live jobs survive while terminal history is pruned and the cap re-admits after compaction.

2. unsafe policy — opened the security-decision issue first, per CONTRIBUTING: Security decision: platform FFI for managed-agent runtime (process identity, Job Objects) #6047. Windows/macOS process-identity (GetProcessTimes, proc_pidinfo) and Job Object FFI is consolidated behind fn-scoped #[allow(unsafe_code)] citing that issue, matching the existing buzz-dev-mcp shell.rs precedent (git_bash_from_registry, KillGroup); stray statement-level allows removed. If there's a safe wrapper crate you'd prefer, name it on Security decision: platform FFI for managed-agent runtime (process identity, Job Objects) #6047 and I'll switch.

3. PR description corrected: GET /jobs + GET /jobs/{job_id}; writes through POST /events.

Verification: cargo clippy -p buzz-runtime -p buzz-acp -p buzz-dev-mcp --all-targets -- -D warnings clean; buzz-runtime lib 30✓ (+10 doc/integration across doctests), buzz-acp lib 839✓.

[jlwainwright]

@themiguelamador ready for re-review when you have a moment