feat(desktop): workspace-scoped agent definition store - #4485
Draft
wpfleger96 wants to merge 101 commits into
Draft
feat(desktop): workspace-scoped agent definition store#4485wpfleger96 wants to merge 101 commits into
wpfleger96 wants to merge 101 commits into
Conversation
Introduce the `WorkspaceAgentScope` type and the scaffolding for a
four-stage workspace transition state machine (Phase 1 foundation).
## Scope model (managed_agents/scope.rs)
- `WorkspaceAgentScope { scope_id, relay_url, owner_pubkey, definitions_dir,
generation }` — the single scope authority for a workspace's agent
definition store. Immutable; callers capture one scope at operation
entry and thread it through `_at(scope)` APIs.
- `derive_scope_id(relay_url, owner_pubkey)` — the canonical sha256
derivation, byte-identical to the retention DB derivation. Both
subsystems now go through one shared helper so "same scope" can never
disagree between definitions and retention.
- `next_scope_generation()` / `current_scope_generation()` — global
monotonic counter incremented on every scope change or identity-import
clear. Long-running operations read at entry and revalidate before
commit; a stale commit aborts.
- `WorkspaceApplyResult { applied, degraded }` — typed result for the
four-stage transition machine (prepare / drain / commit / post-commit).
- Scoped layout: `agents/scopes/<scope_id>/{managed-agents.json,
teams.json, global-agent-config.json}`.
## AppState additions (app_state.rs)
- `identity_mutation: AsyncMutex<()>` (was `Mutex<()>`) — Layer 1 async
lock; callers may `.await` while holding it. Converted so the workspace
transition machine can hold it across awaits without blocking the
executor.
- `workspace_transition: AsyncMutex<()>` — serializes workspace
transitions (`apply_workspace` and live identity import). Lock order:
identity_mutation → workspace_transition → Mesh rearm → mesh_llm_runtime.
- `active_agent_scope: Mutex<Option<WorkspaceAgentScope>>` — `None` from
boot until the first successful `apply_workspace`. Every agent command
fails closed on `None`; there is NO fallback to the legacy unscoped root.
## Retention parity (retention.rs)
- `scoped_retention_db_path` now delegates to `derive_scope_id` instead
of inlining its own sha256, making the hash provably identical.
- `scope_for_arrival` / `arrival_retention_scope` extended to match on
both relay AND owner pubkey. An in-flight old-owner event on the same
relay can no longer land in the new owner's active store after an
identity switch.
## Inbound reconcile (commands/personas/inbound.rs)
- Both `arrival_retention_scope` call sites pass the event's pubkey as
the owner dimension, closing the identity-switch cross-contamination gap.
## Caller updates
- `identity.rs`: three `identity_mutation.lock().map_err()` callers
converted to `.blocking_lock()` (Tokio async mutex's sync-context
variant, safe from `spawn_blocking` threads).
- `identity_key_backup_tests.rs`: test thread mirror updated to match.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…chokepoints ## Scoped storage APIs Add path-based `_at(definitions_dir)` variants alongside every `app: &AppHandle` storage chokepoint. These are the primary API for long-running operations that have captured a `WorkspaceAgentScope` at their entry point: - `storage.rs`: `load_agent_store_at`, `load_managed_agents_at`, `load_agent_definitions_at`, `save_managed_agents_at`, `save_agent_definitions_at`, `managed_agents_store_path_at`; the internal `write_agent_store` now delegates to `write_agent_store_to_path` which is shared with the new scoped write path. - `teams.rs`: `teams_store_path_at`, `load_teams_at`, `save_teams_at`. - `global_config/mod.rs`: `global_config_path_at`, `load_global_agent_config_at`, `save_global_agent_config_at`; the load path is factored into `load_global_agent_config_from_path`. ## AppState scope helpers - `capture_active_scope()` — snapshot of current `Option<WorkspaceAgentScope>`. Callers crossing `.await` or thread boundaries capture at entry. - `commit_active_scope(scope)` — infallible commit-stage setter (Layer 2). - `clear_active_scope()` — clear + generation bump for identity import drain and prepare-stage rollback. ## Event-sync retarget `run_event_sync`, `spawn_event_sync`, `migrate_personas_to_events`, `migrate_teams_to_events`, and `reconcile_agents_to_events` all gain a `definitions_dir: &Path` / `PathBuf` parameter. They no longer resolve the base dir from `AppHandle` — the caller passes the scoped definitions dir directly, closing the bypass that read from the legacy unscoped root. ## apply_workspace scope commit After applying relay + keys, `apply_workspace` derives a `WorkspaceAgentScope` from the effective (relay, owner) pair and commits it via `commit_active_scope`. The immediately following `spawn_event_sync` call reads the committed scope via `capture_active_scope()`, so event sync for this apply uses the scoped definitions dir. A legacy-root fallback is preserved during the Phase 1→2 transition period for pre-apply boot callers. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…mantics Three storage chokepoints (managed_agents_store_path, teams_store_path, global_config_path) now route through capture_active_scope() and fail with a clear error when no workspace scope is active. There is no fallback to the legacy unscoped root — returning a legacy path would recreate split-brain storage. apply_workspace acquires the workspace_transition lock (Layer 1 async serialization) before entering spawn_blocking so scope transitions are serialized against concurrent import_identity calls. import_identity implements both scope modes per v4 plan: - No-active-scope path (recovery/onboarding): persist identity, clear scope, bump generation. No scope is derived or claimed; the next apply_workspace performs adoption. - Live-active path (membership-denied flow): drain managed-agent runtimes (delegates to shutdown_managed_agents), persist identity, clear scope, bump generation. Drain failures are logged but non-fatal; the frontend's re-apply restores agents. Both paths bump the scope generation so in-flight operations see a new generation and abort their commits. The fallback relay can never claim legacy data — claims are only written inside apply_workspace's prepare stage. All 2112 tests pass. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Convert restore.rs, shutdown.rs, and list_managed_agent_runtimes to use a single captured workspace scope per logical operation rather than re-resolving the active scope on every load/save call. restore.rs: - backfill_persona_snapshots captures scope at entry; uses load_managed_agents_at / save_managed_agents_at / load_personas_at throughout the single store-lock epoch. - restore_managed_agents_on_launch captures scope at function entry and clones definitions_dir; all three phases (A: collect, B: spawn, C: write-back) use the same captured path, preventing a concurrent workspace switch from writing Phase C results into the wrong scope. - persist_restore_error receives definitions_dir explicitly. - Both functions return Err (with a clear message) when no scope is active, keeping the fail-closed invariant. shutdown.rs: - When no workspace scope is active (boot before apply_workspace, or after import_identity cleared the scope) skip load_managed_agents and drain only from the in-memory runtime map. Prevents the shutdown path from panicking with 'no active workspace scope'. - record_idx: Option<usize> on AgentToStop distinguishes runtimes with a backing record from those drained without one. - save_managed_agents only called when records were actually loaded. runtime_commands.rs: - list_managed_agent_runtimes captures scope at function entry and uses load_personas_at / load_global_agent_config_at / load_managed_agents_at / save_managed_agents_at so all reads in one poll see the same scope, even if a workspace switch races between the pre-lock and in-lock loads. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…m ledger Implements the Phase 2 scope initialization pipeline: - scope_init.rs: staged directory install with durable manifest (AdoptedLegacy | LegacyClaimedByOther | FreshNoLegacy), atomic rename, separate _ready marker, crash-safe restart semantics - Canonical family claim ledger: reads retention.db's retention_migrations table first (pre-existing claims win); falls back to agents/legacy-claim.json when no retention.db exists - Legacy adoption: copies managed-agents.json, teams.json, global-agent-config.json, and personas.json (when present) into a sibling ._staging directory, then renames atomically - apply_workspace: Prepare stage now calls ensure_scope_ready before the Layer-2 commit epoch; a failed prepare leaves the old scope active and untouched - 6 new unit tests cover FreshNoLegacy, AdoptedLegacy, second-scope LegacyClaimedByOther, idempotent re-init, staging cleanup on retry, and retention.db claim taking precedence over first-activation order All 2118 tests pass. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ped dev-sync Completes Phase 2 of the workspace-scoped agent store: - scope_init.rs: replace the run_scoped_migrations no-op with the full ordered migration pipeline (fold, strip, refresh-avatars, backfill, detach, reconcile-names, reconcile-mcp, databricks-v1-to-v2, materialize) running against the scope directory after staged install. Ordering mirrors migration.rs::run_boot_migrations_inner's load-bearing order. - migration submodules: expose fold_personas_in_dir, strip_baked_team_ instructions_in_dir, backfill_standalone_agents_in_dir, detach_directory_ backed_teams_in_dir, materialize_runtimes_in_file as pub(crate) - migration.rs: add _at(definitions_dir) wrapper functions for reconcile_ provider_mcp_commands, reconcile_databricks_v1_to_v2, refresh_builtin_ agent_avatars, reconcile_legacy_command_names, materialize_agent_runtimes re-export the dir-level helpers under the crate's migration module - SHARED_AGENT_FILES: emptied; legacy unscoped files no longer symlinked across worktrees (they live under agents/scopes/ now) - SHARED_AGENT_DIRS: add agents/scopes so all scoped stores are shared across dev worktrees without requiring knowledge of the dynamic scope ID - migration_tests.rs: rewrite 8 sync tests to match the new SHARED_AGENT_DIRS layout; add scope-dir-based write-through and seed-up tests All 2118 tests pass. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Phase 3 of workspace-scoped agent store:
3a: reconcile_managed_agent_runtimes loses its `communities` parameter.
The backend derives the sole target relay from the captured active scope;
cross-scope fan-out is no longer representable at the API level.
- runtime_commands.rs: capture active scope relay; remove communities Vec
- runtime_types.rs: remove ManagedAgentCommunityTarget struct
- tauriManagedAgents.ts: reconcileManagedAgentRuntimes() takes no args
- managedAgentRuntimeHooks.ts: bootstrapManagedAgentRuntimePairs calls
parameterless reconcile; drop communities list construction
- useManagedAgentRuntimeReconciliation.ts: rewritten to track a single
activeCommunityKey instead of per-relay state; simplified retry logic
- AppShell.tsx: pass `${activeCommunity?.id}-${reinitKey}` as the key
3b: Mesh relay-match reuse rule + fail-closed serve preflight + watchdog.
- mesh_llm.rs: ensure_relay_mesh_for_record captures scope relay at entry;
a live runtime is only reused when its relay matches the scope relay;
serve-mode mismatch fails closed with a precise 'Share Compute is
currently pinned to <relay>' error; client-mode mismatch falls through
to re-arm; drain_mesh_client_if_stale drains a client whose relay
differs from the incoming workspace relay (Layer-1 async, non-fatal).
- recovery.rs: rearm_relay_mesh_for_running_agents captures one scope per
pass; Live early-return only taken on relay match; serve-mode Live
mismatch skips the pass (machine-level pinning).
- personas.rs: add scoped load_personas_at / save_personas_at variants.
3c: Drain journal + compensation + apply_workspace rewrite.
- runtime_commands.rs: DrainJournalEntry struct, drain_scope_runtimes
(snapshot journal + stop all live runtimes, returns stopped/remaining/
first_error), compensate_drain (restart exactly the stopped entries).
- workspace.rs: apply_workspace return type changed from () to
WorkspaceApplyResult. Layer-1 async drains the Mesh client before
spawn_blocking. Drain stage acquires managed_agent_runtime_transition,
calls drain_scope_runtimes; on failure calls compensate_drain and
returns applied:false. Per-transition restore replaces the launch-only
managed_agent_restore_pending one-shot. Post-commit failures (event
sync, restore) surface as degraded entries on WorkspaceApplyResult.
3d: Scope-tagged runtime map entries.
- runtime_types.rs: ManagedAgentPairRuntime gains scope_id: Option<String>
- starting() constructor takes scope_id; captured from active scope at
spawn time in runtime_commands.rs, restore.rs, and runtime.rs.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…Phase 3e)
Surfaces the degraded-apply result from the backend all the way to the UI:
tauri.ts:
- Add WorkspaceApplyResult interface { applied: boolean; degraded: string[] }
- applyCommunity() now returns Promise<WorkspaceApplyResult> instead of
Promise<void>; passes typed result through from apply_workspace command.
useCommunityInit.ts:
- Consumes applyResult from applyCommunity instead of discarding void.
- applied: false (drain-failed) → park on the loading gate so the user
can retry via a workspace switch; the specific degradation messages are
shown as the error.
- applied: true with degraded entries → console.warn (informational;
workspace IS active; post-commit step failed gracefully).
- Existing catch block still handles genuine Tauri errors (poisoned lock,
invalid nsec, etc.) unchanged.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
ManagedAgentPairRuntime::starting() now takes a scope_id argument. Update the test helper that constructs a fake PairRuntime to pass None. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…rnal extraction Extract execute_drain_journal() as a pure inner function that takes the runtime HashMap directly — no AppHandle needed — enabling deterministic unit testing of the drain/compensate logic without a Tauri mock app. Move the test block from runtime_commands.rs to the sibling runtime_commands_tests.rs (following the storage_tests.rs pattern) to keep the main file under the 1000-line size gate. New tests: - test_drain_empty_map_returns_success - test_drain_exited_process_counts_as_stopped_and_clears_map - test_drain_scope_id_propagates_from_runtime_starting - test_drain_missing_key_treated_as_already_stopped - test_drain_cleanup_fn_called_for_each_stopped_entry - test_workspace_apply_result_drain_failed_returns_applied_false - test_workspace_apply_result_degradation_accumulates All 2125 existing tests continue to pass. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…y tests Add 13 missing Phase 4 unit tests covering the full v4 test matrix: Scope model (scope.rs): - test_generation_staleness_detected_after_scope_change — stale-commit detection: captured generation G diverges from current after a switch - test_scope_switch_a_to_b_to_a_advances_generation — A→B→A round-trip produces strictly increasing generations; relay fields correct at each step - test_rapid_scope_switch_a_b_c_all_stale_after_c — rapid A→B→C: both A and B stale relative to C; counter order A < B < C - test_switch_during_restore_detected_by_generation_check — mid-flight switch detected by generation check without spawning threads Identity/scope lifecycle (app_state_tests.rs): - test_import_before_first_apply_leaves_scope_none — import when scope=None does not derive/claim any scope; only bumps generation - test_live_import_with_active_scope_clears_scope_and_bumps_generation — live import clears scope and advances generation; commands fail closed - test_fallback_relay_never_claims_during_identity_import — identity import operations (clear + bump) never touch the filesystem claim ledger - test_prepare_failure_leaves_old_scope_intact — old scope unchanged when commit_active_scope is never called (prepare error path) - test_inactive_runtime_exit_after_scope_cleared_is_safe — scope=None after clear is safe for runtime-exit observers Crash boundaries (scope_init.rs): - test_crash_after_claim_before_staging_resumes_correctly — fallback claim exists, no staging: full staged install runs, legacy adopted - test_crash_during_staging_copy_is_cleaned_on_retry — stale staging with partial content is cleaned; final file comes from legacy source - test_crash_after_staging_manifest_before_rename_resumes_correctly — staging with manifest but no rename: cleaned and re-run - test_crash_after_rename_before_ready_resumes_migrations — target exists with manifest but no _ready: skip re-staging, resume migrations, preserve post-crash writes Also fixes ensure_scope_ready to implement the plan's "installed-but-not-Ready resumes migrations" contract: when the target directory already has a manifest (rename completed), skip install_staged and go straight to migrations + ready marker, preserving any post-crash inbound/interactive writes. Mesh relay-scope (mesh_llm_tests.rs): - test_serve_pinned_relay_mismatch_fails_closed — relay mismatch detection + fail-closed error prefix verified against the exact code path - test_client_relay_mismatch_is_not_fail_closed — client mismatch falls through (treat as absent), not the serve fail-closed error - test_watchdog_scope_relay_check_uses_normalized_comparison — relay normalization consistency including trailing-slash and whitespace edge cases All 2138 tests pass (was 2125). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Nine files were over the gate limit after the workspace-scoped store implementation. Trims/extractions to get all under limit: - app_state_tests.rs: move scope lifecycle tests to app_state_scope_tests.rs - app_state.rs: move pending_owned_channels methods to identity_storage.rs - AppShell.tsx: inline reconciliation key as String(reinitKey) (1 line vs 3) - tauri.ts: type alias ApplyWorkspaceResult + biome-ignore format to keep the applyCommunity body under the limit - runtime.rs: restructure scope capture to save a net line - storage.rs: trim doc comments on _at variants to single-liners - mesh_llm.rs: make scope_impl pub(crate) mod; fold scope relay capture inside check_mesh_runtime_relay_scope; remove verbose comments - migration.rs: extract scoped migration helpers to migration_scope.rs via include!(); trim SHARED_AGENT_FILES/DIRS block comments - migration_tests.rs: trim explanatory comments to save net 33 lines Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…mmit (C3) Lock architecture fix: `managed_agent_runtime_transition` is now held from journal creation through the end of the commit swap so no concurrent start/reconcile can insert a runtime in the gap between drain and scope publication. All fallible commit guards (relay_url_override, keys, active_agent_scope) are acquired BEFORE any field is mutated. A lock-poison failure after drain runs compensation and returns `applied: false` — never a half-committed state. The prior code dropped the transition guard at the end of the drain block (inner scope) while the adjacent comment claimed "the commit below also holds it" — the comment was false. Removes that false claim. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…d migrations, atomic drain, import drain protocol, boot migration strip) C1: Remove #[allow(dead_code)] from WorkspaceAgentScope owner_pubkey and generation fields; add validate_scope_generation() production helper; restore Phase B uses captured scope relay (not live relay_ws_url_with_override); Phase C validates generation before acquiring store lock and terminates stale-spawn children. C2: run_scoped_migrations returns Result<(), String> propagating first failure; ensure_scope_ready withholds _ready on Err; added Step 10 JSON validation gate; fixed crash-resume test fixture to use valid JSON; added migration-failure test that verifies no _ready on corrupt input, then repair+retry writes _ready. C3: (Already committed as 3325363.) Transition lock held continuously from drain through commit. C4: drain_managed_agent_runtimes_for_import returns Result<Vec<DrainJournalEntry>>; import_identity acquires managed_agent_runtime_transition lock for live-active path; drain failure compensates and returns Err before identity persist; persist failure compensates stopped entries and returns Err; removed commit_active_scope from identity_storage.rs. C5: run_boot_migrations_inner stripped of all definition-touching steps (now in scoped pipeline); backfill_persona_snapshots_at added and called in prepare stage; legacy retention migration moved to prepare stage; try_regenerate_nest removed from lib.rs boot (now post-commit in workspace.rs); managed_agent_restore_pending field and write removed from AppState and lib.rs. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ecovery from captured WorkspaceAgentScope active_retention_scope now derives relay and owner from capture_active_scope() rather than reading relay_ws_url_with_override + signing_keys() independently. Returns Err when no active scope exists (fail closed) or when signing keys pubkey disagrees with scope owner (defensive guard). rearm_relay_mesh_for_running_agents captures both relay and definitions_dir from the active scope at function entry. All store reads (load_managed_agents, load_personas, load_global_agent_config) and error-persist writes now use _at(definitions_dir) so they target the captured scope's store throughout the recovery pass, not whichever scope happens to be active when each helper runs. persist_mesh_last_error and clear_mesh_last_error_if_set refactored to _at() variants that take an explicit definitions_dir rather than resolving through the live active scope. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…pShell composite key
spawn_event_sync returns Result<(), String> so dispatch failure can be captured
by the workspace-apply post-commit section rather than being silently ignored.
The value is always Ok(()) since tauri::async_runtime::spawn is infallible; this
establishes the typed interface for future signaling.
try_regenerate_nest returns Result<(), String> instead of swallowing errors.
All fire-and-forget callers updated to .ok() to explicitly discard the Result.
apply_workspace post-commit:
- try_regenerate_nest moved out of the spawn_blocking closure into the async
post-commit section so its Result can populate the degraded vec.
- spawn_event_sync Result captured; dispatch failure pushed to degraded.
- Nest failure reported as 'nest context regeneration failed: ...' degradation.
useCommunityInit.ts: post-commit degraded items now emit a toast.warning (8 s)
via sonner so the user sees partial failures. Previously only console.warn.
AppShell.tsx: useManagedAgentRuntimeReconciliation key changed from
String(reinitKey) to `${activeCommunity?.id}-${reinitKey}`. A same-relay
identity swap (new communityId, unchanged reinitKey) now correctly re-triggers
runtime reconciliation. Destructured activeCommunity and reinitKey from
communitiesHook and updated two other call sites for consistency.
dead pub use exports in migration.rs removed (fold, backfill, detach, strip,
materialize — all now accessed only through scoped _in_dir/_at variants).
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
C4 removed commit_active_scope from identity_storage.rs (no longer called in production after the inline commit). app_state_scope_tests.rs uses it as a test helper to set up a live scope without running the full apply_workspace pipeline. Re-add it under #[cfg(test)] so tests continue to compile. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ncile key Without the '?? "none"' fallback, a null activeCommunity?.id produces the string "undefined-N" rather than "none-N". Both are valid change signals, but the explicit fallback matches the existing pattern at line 233 of the same file and satisfies Thufir's pass-2 finding C7. The line expands to 3 lines after biome formatting (88 chars), landing AppShell.tsx at exactly 1000 gate-counted lines — still within the 1000-line ratchet (gate condition is > 1000). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
After C5 stripped the per-scope definition migrations from the global boot path, the app-level wrapper functions (fold, materialize, backfill, detach, team_suffix, refresh_builtin_agent_avatars, reconcile_*) became unused. Their scoped _at()/_in_dir() variants are what the pipeline calls. Add #[allow(dead_code)] with a rationale comment to each wrapper rather than deleting them — the wrappers document the prior call shape and serve as reference for future integration. Also drop the spurious let _ = binding on remove_agent_runtime_receipt (returns (), not Result) flagged by clippy::let-unit-value. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Adds test_two_workspace_relay_partition to the managed-agent e2e suite. The test models workspace A and workspace B as two distinct owner keypairs on the same relay and verifies in both directions: 1. Owner A's NIP-33 author-scoped subscription returns only A's definition, not B's (workspace B content never leaks into A's view). 2. Owner B's subscription is symmetric — returns only B's definition. 3. Cross-scope queries prove NIP-33 (kind, author, d-tag) scoping: two owners publishing under the same d-tag get distinct relay coordinates that cannot collide or bleed across. This is the relay-level half of the live two-workspace leak probe required by the workspace-scoped agent definition store (PR #4485, plan v4 Phase 4). The filesystem-level half is covered by scope_id unit tests confirming that distinct (relay_url, owner_pubkey) pairs always produce distinct scope_id directories under agents/scopes/<scope_id>/. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
C1 — Production agents-root contract: - scope_init.rs already had the correct base_dir contract (no extra 'agents' join); added production-shaped adoption test that mirrors the exact managed_agents_base_dir semantics to prevent regression. C2/C3 — Deadlock removal + store lock: - workspace.rs: drop rt_transition + store lock BEFORE compensate_drain on all three commit-guard failure paths; hold store lock from drain through commit so concurrent store writes cannot interleave. - identity.rs: drain returns Err((stopped, msg)); compensate uses the real stopped slice, not []; locks dropped before compensate_drain. C4 — Captured-scope completion: - confirm_team_snapshot_import and confirm_agent_snapshot_import: both now capture scope at entry, use _at() APIs throughout, validate generation before first write, resolve RetentionScope from captured. - Mesh recovery (recovery.rs): capture full WorkspaceAgentScope at entry; validate generation before each write to definitions_dir. - Restore missing-record stale-child: when find_managed_agent_mut fails for a spawned child (record deleted between Phase B and C), terminate the child and remove its receipt instead of leaking the process. C5 — Pre-Ready family in scope initializer: - ensure_scope_ready gains owner_pubkey parameter. - New run_pre_ready_family: runs legacy retention migration and persona snapshot backfill before writing _ready so a crash leaves the scope in a retryable state, not permanently marked Ready with incomplete data. - workspace.rs guards remain for pre-existing Ready scopes (idempotent). C6 — Delete dead boot-migration wrappers: - Deleted backfill_standalone_agents, detach_directory_backed_teams, and strip_baked_team_instructions (the #[allow(dead_code)]-suppressed app-level wrappers); their _in_dir equivalents are the authoritative scoped pipeline entry points. - Removed tests for the deleted functions from migration_command_tests.rs. C7 — Structured degradation reporting: - spawn_event_sync return type changed from Result<(), String> to (): the dispatch cannot fail; the false Result contract is removed. - workspace.rs restore spawn now emits workspace-degraded Tauri event when restore_managed_agents_on_launch returns Err, making restore failures observable to the UI instead of silently logged. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…tions - backfill.rs: remove blank line between two consecutive doc comment blocks - detach.rs: merge orphaned step-list doc comment into function doc comment - migration.rs: remove blank line after doc comment before private fn - migration_tests.rs, migration_command_tests.rs, migration_avatar_tests.rs, migration_databricks_tests.rs: add .unwrap() to calls that now return Result after C5 migration fallibility changes Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Contributor
|
workspace-scoped agents/stores is the right cut for the cross-relay leak. the two-owner e2e note in the test comment helped me follow the nip-33 half. |
…y, captured scope, lock-aware compensation
Eight corrections from the resumed loop (fresh 3-pass budget, Option A ruling):
1. Option A Mesh: delete pre-prepare drain_mesh_client_if_stale and rollback
restore_mesh_sharing (compensated a drain that no longer happens). Replace
with fail_if_client_mesh_active preflight in both apply_workspace and
identity import. Journaled Mesh recipe deferred as tracked follow-up.
2. Versioned _ready: scope_is_ready now reads marker content and compares
against READY_MARKER_VERSION ("v1"); old unversioned markers return false
and force re-run through run_pre_ready_family. Delete log-only post-ready
best-effort guards (backfill + retention migration) from apply_workspace.
Add test: old marker -> pipeline re-runs -> version advances.
3. Snapshot outbound phases use captured scope relay: both
confirm_agent_snapshot_import (Phase 3b profile) and
confirm_team_snapshot_import (Phases 4/5 profile + memory) now use
captured_scope.relay_url instead of relay_ws_url_with_override.
Add test proving outbound relay is captured-scope, not live-state.
4. Generation checks atomic with writes: global_agent_config Phase 1 validates
scope generation inside the store lock before writing config; Phase 2
(restart_local_agent_on_config_change) validates under lock before stop.
collect_restart_candidates renamed to collect_restart_candidates_at with
definitions_dir parameter. Mesh recovery helpers (persist_mesh_last_error_at,
clear_mesh_last_error_if_set_at) take captured_scope and validate generation
inside the store lock.
5. Lock-aware compensation gate: AtomicBool
managed_agent_drain_compensation_in_progress added to AppState.
compensate_drain sets it true (Release) before restarting entries, false
after. start_pair loads it (Acquire) before taking the transition lock and
returns Err if set. Closes the drop-then-compensate interleave window
without recursive locking. Add deterministic partial-drain test.
6. Pre-scope migrations deleted: migrate_agent_keys_to_dev_service (AppHandle
variant) removed from storage.rs. Pre-scope calls removed from
run_boot_migrations_inner. Scoped variants in run_pre_ready_family are
authoritative.
7. Degradation wired to UI: workspace-degraded Tauri event listener added to
useNestNotifications.ts (toast.error with payload as description). False
comment about emit_workspace_degradation removed from event_sync.rs.
backfill_persona_snapshots_at (dead lock-taking wrapper) deleted.
8. e2e test docs: test_two_workspace_relay_partition comment corrected --
Direction 3 asserts len==1 (B's event), not zero. Explicit note added that
this test does not cover desktop workspaces or substitute for the live probe.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…t.rs) Move scope_init tests to scope_init_tests.rs via #[path] include to bring scope_init.rs under the 1000-line ratchet (603 lines after extraction). Move test_outbound_relay_uses_captured_scope_not_live_state from import_avatar_tests (in import.rs) to the adjacent tests.rs to bring import.rs under the 1000-line limit (999 lines after move). Both files previously crossed the limit after the resume-pass corrections added the versioned-ready test block and the captured-relay test. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…h stop, scope, CI
Item 1 — Compensation primitive: replace AtomicBool gate with lock-owning
compensate_drain that takes the caller's already-held rt_transition guard
by value, re-acquires only the store lock, validates captured scope generation,
then restores journal entries via start_pair_under_held_locks. Split
start_pair_under_held_locks out of start_pair so both the normal and
compensation paths share the spawn-and-register body. execute_drain_journal
refactored to accept an injectable stop_fn via drain_journal_with_stop;
execute_drain_journal_with_stop_fn exposed for test injection. Tests: live
SIGKILL drain, structural lock-release proof, deterministic partial-failure
test with injected stop error covering stopped prefix and remaining tail.
Item 2 — Client stop + start serialization: mesh_stop_client Tauri command
in mesh_llm_scope.rs stops only a client-mode runtime; serve/absent are
no-ops. Client start (ensure_relay_mesh_for_record) acquires
workspace_transition through runtime installation to serialize against
apply_workspace, which holds workspace_transition from before the Option A
preflight through commit. fail_if_client_mesh_active preflight runs under
workspace_transition so no new client can start in the check→commit gap.
UI: 'Stop using shared compute' button in MeshComputeSettingsCard shown when
isConsuming; calls new meshStopClient() in tauriMesh.ts. e2eBridge mock for
mesh_stop_client added.
Item 3 — Fallible migrations + atomic marker: rename_provider_to_runtime_in_personas
propagates Result; migrate_agent_keys_to_dev_service_at returns Result and
propagates from copy_agent_keys_between_stores. run_scoped_migrations uses ?
on persona-provider step. _ready marker written via temp+rename (atomic).
dev-key migration skipped in unit-test builds (#[cfg(not(test))]) to avoid
macOS Keychain dialogs. Tests: old-marker upgrade (no scope deletion), partial
migration failure withholds v1 until repair succeeds.
Item 4 — Global-config captured respawn: Phase 2 restart validates captured
scope generation under store lock before stop, and again before respawn via
start_local_agent_pairs_with_preflight_at (new captured variant using
definitions_dir). persist_last_error validates generation under store lock.
Item 5 — Snapshot imports captured operation context: both confirm_agent_snapshot_import
and confirm_team_snapshot_import capture owner keys at entry, verify against
captured_scope.owner_pubkey immediately, thread captured keys through all
mint/retention/engram phases. Re-verify owner key under store lock before
Phase 3a write. Outbound profile/memory phases use captured_scope.relay_url.
Item 6 — CI red: rustfmt applied (agents_scoped.rs, import.rs); clippy
needless_borrow at team_snapshot.rs:793 fixed; e2eBridge apply_workspace mock
returns { applied: true, degraded: [] } in both immediate and delayed branches;
mesh_stop_client mock case added. Stale 3-line doc fragment removed from
mesh_llm.rs; visibility of re-exported agents_scoped fns bumped to pub(crate).
Item 7 — Listener behavioral test: useNestNotifications.test.mjs exercises
workspace-degraded toast payload, unlisten cleanup, and boundary payloads
without requiring a real Tauri runtime. Doc comment updated: event-sync dispatch
failure does not emit workspace-degraded (shutdown-time, no toast surface).
Item 8 — Dead code (minor): backfill_persona_snapshots AppHandle shim removed;
stale scope_init.rs:388-393 comment corrected; make_base_dir test helper removed.
File-size gate: mesh_llm.rs (999), import.rs (999), team_snapshot.rs (999).
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Resolves two conflicts against main:
AppShell.tsx: HEAD had const { activeCommunity, reinitKey } = communitiesHook
for the composite workspace key; origin/main added useHuddlePresentation()
destructuring from the Huddle redesign (#4281). Resolution keeps both: the
composite key is required for useManagedAgentRuntimeReconciliation, and the
Huddle hooks are needed for the new Huddle UI.
MeshComputeSettingsCard.tsx: HEAD had the Stop using shared compute affordance
plus the legacy inline model section; origin/main (#3735) replaced the inline
model section with the MeshModelPicker component. Resolution keeps the Stop
button block and adopts the MeshModelPicker layout, discarding the replaced
inline model controls.
Also corrects the false comment at runtime_commands_tests.rs:342-345 that
claimed compensate_drain is covered by the desktop integration test suite.
The compensation round-trip requires an AppHandle; the codebase has no
tauri::test harness and no AppHandle mock. The honest coverage statement is:
drain-prefix contract proven by unit test, restart path integration-only.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Add OwnerIdentityCapability<P> over the existing egress registry: a generation-stamped, registry-tracked handle for authority that outlives the bounded lease that derived it. Two policies land: Session (authenticated connections — the huddle audio socket, later the frontend relay WS) and Bearer (pre-minted Blossom headers, threaded in a follow-up). Each capability is registered with its revocation handle (the session's cancellation token; the bearer's registry id) so the C5 coordinator barrier only invokes what C2 registered — it never retrofits the registry schema. admit_exercise() validates BOTH current egress admission AND capability_generation == current identity-persistence generation immediately before each transmission, so a stale capability sends zero bytes. Issuance runs under a bounded lease (the signing that derives the capability is an ordinary leased operation). The huddle audio socket is threaded: the NIP-42 auth signs under a bounded lease (dropped before the joined-await), the session capability is registered with the connection's cancel token, and the send task validates it before every frame batch — a frame cannot ride the established peer after an identity transition supersedes it. C2 builds substrate only: the coordinator revocation barrier (revoke_durable_capabilities_before) and drain wiring defer to C5 with the egress drain, gated behind the same generation bump C5 introduces. Per-item allow(dead_code) with the C5-consumer comment; C5's zero-allow confirmation extends to these. generation never bumps until C5, so this is behavior-preserving. 8 new unit tests (2438 lib pass): generation-stamp, exercise admits when live+current, stale-capability zero-bytes controls (generation bump, drain, latch) for both kinds, barrier revokes old-generation only, registration- completeness + deregister-on-drop, and a no-transition control. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…auth (C2b) Bring the owner-derived Blossom bearers into the durable-capability world so a header minted under identity A cannot be attached after a transition to B. mint_media_get_auth and the do_upload t=upload mint now sign under a bounded egress lease (issuance is an ordinary leased operation, spec L4569-4570) and return/hold an OwnerIdentityCapability<BearerPolicy> registered with its revocation handle. The four get-auth attach sites (media_download, personas::card, media_proxy x2) and the upload dispatch validate the bearer via admit_exercise() immediately before the HTTP send — a stale capability attaches nothing (get-auth stays fail-open) or uploads zero bytes. mint_media_get_auth becomes async; the ripple is a mechanical .await through its four already-async callers. Removes the register_owner_bearer allow(dead_code) now that C2b consumes it. Substrate coverage is unchanged: the stale-bearer zero-bytes and registration-completeness controls already pin the exercise behavior these guards depend on. 2438 lib tests pass, clippy + fmt clean. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…x round) Addresses Paul's four C2 findings before C3. F1 (barrier bypass): register_owner_session/register_owner_bearer re-read the current generation at registration, so a capability derived from a losing identity could be stamped with the winning generation and survive the C5 barrier. Both now take &OwnerIdentityEgressLease as a compile-enforced witness and stamp lease.generation() via a shared register_durable() helper — a bump between admission and registration leaves a stale stamp that the first admit_exercise refuses (fail-closed). Huddle registers the session while the auth lease is still held (before the joined await, not spanning it). Corrects the DurableRegistry "same lock" and media.rs "no bump can slip" doc claims. Adds a red-then-green control: admit under gen N, drain bumps to N+1, register, exercise refuses. F3 (doctrine): the upload legacy retry re-attached the signed header without a second admit_exercise. Revalidate before the retry dispatch — two transmissions, two validations. F4 (scope): create_auth_event (frontend relay WS) now signs under a per-send bounded lease, gating reconnection against an in-flight drain. Frontend session REGISTRATION (capability + native-WS teardown) is explicitly deferred to C6/C7 with the frontend identity store. F2 (gates): split six files under the desktop file-size ratchet — never trimmed. owner_identity_egress.rs → directory module with the durable substrate in durable.rs; relay.rs/media.rs/identity.rs move test modules to sibling _tests.rs files; card.rs extracts the card-archive cluster to card/archive.rs and messages.rs the feed-item projection to messages/feed_item.rs. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Extend the durable owner-identity capability registry with ArtifactPolicy
(NIP-AP, P30/P31/P32-C1): a generation-stamped owner-key-derived value that
leaves its producing lease and is applied later. Unlike Session/Bearer,
an artifact owns no side-effecting teardown, so its only revocation is the
exercise-time / application-site generation compare — the transition bump
IS the invalidation (shape (ii), acked by Paul). revoke_durable_capabilities_before
performs no per-entry artifact work; the barrier reaches artifacts via the bump.
The seven commands/identity producers admit a bounded lease before the owner-key
sign/encrypt/decrypt and return the value wrapped as StampedArtifact<T>
({value, artifact:{id,generation}}), stamping the issuing lease's generation
(the F1 lesson). The wire shape is locked on both sides; a serialization test
pins it so C6/C7 inherits a stable contract.
The 7 TS adapters unwrap .value at the boundary with a named C6/C7 deferral —
outward threading to application sites + the generation-compare lands in C6/C7
coupled to the code that reads the stamp (supersedes condition 2 of the
stamp-at-boundary ruling, revised on the measured 87-file transitive radius).
StampedArtifact type and the four identity adapters split into small modules to
respect the desktop file-size ratchet.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Compose main's post-C3 changes with the workspace-scoped agent store
(WSA) refactor, keeping BOTH sides' semantics. Non-mechanical resolutions
(per Paul's cluster ruling, thread eb3246b2):
- Pollen rename → scoped-pipeline step 1.5 (NOT global pre-scope): the
live store on HEAD is scopes/<id>/managed-agents.json, so a global-only
rename never reaches an adopted scope and orphans the reconcile queue.
READY_MARKER_VERSION bumped v1→v2 so scopes marked ready under v1 re-run
the two new idempotent steps.
- Team-membership repair → scoped step 4.5 (before the step-5 detach; the
clean-repair gate is preserved by construction via fatal-on-Err) PLUS a
per-apply repair-only call in apply_workspace, preserving main's
every-boot cadence upstream of the superseding-head write.
- Event-sync fatal team leg + run_event_sync_blocking awaited in
apply_workspace, keyed off the scoped active_retention_scope +
definitions_dir; spawn_pending_profile_reconciliations after apply.
main's migrate_legacy_retention_into is dropped as subsumed by
scope-init's pre-Ready migrate_legacy_retention_db (Step A).
- inbound.rs apply composed HEAD's §2.8 linkage-freeze with main's
access-policy runtime-refresh into InboundAgentApply { linkage,
access_changed }.
- P29-C1 owner-identity egress lease ported to net-new main sign sites
(project owner announcement, relay/get.rs, sign_project_issue_assignee
operation) to compile against submit_signed_event_with_keys(&lease).
Merge fallout resolved: #5682's local-spawn idle-pool-sleep env
(idle_pool_sleep_env / IDLE_POOL_SLEEP_SECS) is subsumed — its only call
site was the local direct-env block HEAD's ff16a80 deleted, and the
remote-deploy policy_env path never carried it; the orphaned symbols are
dropped. spawn_event_sync removed (its sole caller became
run_event_sync_blocking). mesh_llm_tests.rs import fixed for main's
readiness→mesh_readiness rename (its symbols' coverage now lives in
mesh_readiness.rs's own inline tests). Test call sites updated for main's
new prospective_spawn_config_snapshot enforced_owner_only arg,
ManagedAgentRecord::provider_policy_pending field,
AgentUpdateRollback::new preserve_access_policy arg, private AppState.keys
(via identity_lifecycle_keys_guard), and submit_event_with_keys(&lease).
File-size ratchet (base 978e585): four files crossed the 1000-line cap
after the merge and were split, never trimmed. team_snapshot.rs →
team_snapshot/retain.rs (retain_agent_pending); personas/inbound.rs →
inbound/tombstone.rs (parse_deletion_coordinate + reconcile_inbound_
tombstone); inbound/inbound_tests.rs → inbound/team_tests.rs (kind:30176
team-inbound tests); managed_agents/retention.rs → retention/tests.rs
(inline test module).
Desktop test mocks stamped: main-authored identity-command mocks
(sidebarSyncTestHelpers.mjs installTauriMock; communityThemeSync.test.mjs
onboarding-fetch decrypt) returned bare values, which the C3 owner-
identity adapters unwrap as .value → undefined. Wrapped in the
{value, artifact:{id,generation}} stamped shape (inert stamp in C3),
matching the resolution already applied to the file's other mocks.
Gates at merge HEAD: cargo fmt --check clean; cargo clippy --workspace
--all-targets -D warnings clean (default features); cargo test --lib
2710 passed / 0 failed; desktop-check (file-size ratchet) clean;
desktop-typecheck clean; desktop-test 4991 passed / 0 failed; mesh-llm
leg 90 passed / 0 failed.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
get_nsec and create_ncryptsec_backup admit an owner-identity egress lease BEFORE reading/deriving the secret and return their value as a StampedArtifact carrying the issuing lease's generation. The NIP-49 constructor sits outside the sign/encrypt method sweep, so the export class is stamped constructor-agnostically. Adapters unwrap .value; the reveal/copy generation-compare defers to C6/C7. Closed-world enumeration and the ncryptsec source allowlist updated in the same commit. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The production adapters unwrap .value from the StampedArtifact wire shape, but the Playwright bridge mocked sign_event, create_auth_event, nip44_encrypt_to_self, nip44_decrypt_from_self, and sign_nostr_identity_binding with bare returns. At runtime the app under test hit JSON.parse(undefined) at every sign/encrypt site, cascading into membership-subscribe, read-state, and badge failures across the smoke suite. Add a local stamped() helper and route every owner-identity artifact mock (including the C4 nsec/backup cases) through it. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
h2 0.4.14 is vulnerable to unbounded memory growth from empty DATA frames (RUSTSEC-2026-0258, patched in 0.4.16). cargo-deny fails the Security gate on the advisory ingest. Bump is lockfile-only; 224 other dependencies unchanged. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…nator Phone-recovery identity swap (`import_recovered_identity`) bypassed the managed-agent runtime drain, active-scope clear, and scope-generation bump that `import_identity` performs (P25-C1): it acquired only `identity_mutation` and committed via `commit_imported_identity` directly. A recovery in an active workspace left stale agent runtimes bound to the prior identity. Extract the shared `run_identity_transition` coordinator so both callers route through the same lock-ordering and drain path. The pre-commit boundary is generalized to a `commit_under_fence` primitive: it locks the supplied fence, runs the validity check, and runs the durable commit under the same held guard, so a racing supersession can neither interleave nor slip between check and commit (P26-C1). Normal import passes no fence and an always-Ok check; recovery passes the pairing `generation_fence` and the task-currency check. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… module The C5 coordinator additions (`run_identity_transition`, `commit_under_fence`) pushed `commands/identity.rs` past the 1000-line desktop file-size ratchet. Move the coordinator cluster — the drain helper, `run_identity_transition`, `commit_under_fence`, and `import_identity_blocking` — into a `#[path]` sibling `identity_transition.rs`, mirroring the `agents_scoped.rs` split. The two `pub(crate)` entry points are re-exported so external call paths (`import_identity`, phone recovery, the fence tests) are unchanged. No behavior change. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ed-agent-store * origin/main: fix(desktop): eliminate mounted-view CPU burn — compositor-safe shimmer, observer append fast path, poll-tick disk reads (#6198) Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com> # Conflicts: # desktop/src-tauri/src/commands/agents.rs
…ed-agent-store * origin/main: fix(desktop): bind presence retry timers (#6213) ci: make file-size policy a first-class gate (#6187) Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ble outcomes (P27-C1) The base persistence kernel returns a binary Result, but it is not transactional: a returned Err does not prove no durable key write happened. A coordinator that compensates on any Err can leave durable identity B on disk beside live in-memory identity A, and a later reachable-keyring restart activates B (the P25/P26 split-state class). Add persist_imported_identity_classified, which wraps the proven kernel and derives a three-valued PersistenceOutcome from durable fact: Committed (B proven canonical, compensation forbidden), DefinitelyUnchanged (B never landed, the only compensable outcome), and Indeterminate (neither proven, fail closed). On a kernel Err it re-reads the keyring and identity.key under the caller's held transition guards and classifies from what they hold; a keyring unreachable on re-read fails closed. The C5 coordinator consumes this in the barrier commit. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
An in-memory Indeterminate latch does not survive a restart: a process
that returned an ambiguous persistence outcome and then crashed would
boot fully-signable A even though a durable B may exist, and later
resolve to B when the keyring returns.
Add identity_transition_journal: an fsync'd JSON row recording
{from_pubkey, to_pubkey}, written before the coordinator dispatches the
durable B write and cleared only at the two proven exits. read_pending /
pending_exists let startup honor a surviving row, and a present-but-
corrupt row reads as pending (fail closed) rather than absent. The C5
coordinator writes/clears it in the barrier commit and startup honors it
in the recovery-route commit.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ity import (P26-C1) run_identity_transition previously sampled has_active_scope before acquiring workspace_transition and skipped the lock entirely on the no-scope path. That left a None -> Some activation race: a concurrent apply_workspace could commit a new active scope between the pre-branch sample and the durable identity commit, so the import would drain nothing while a live scope existed. Route both the mesh-llm and non-mesh paths through workspace_transition unconditionally, and move the active/no-scope decision inside import_identity_blocking, derived from a scope snapshot taken under the held guard. Sampling under the guard closes the race: apply_workspace and identity import now serialize on workspace_transition, so no activation can slip between the sample and the commit. No deadlock — apply_workspace takes only workspace_transition (never identity_mutation), so the global order identity_mutation -> workspace_transition has no cycle. Drops the now-redundant has_active_scope bool from import_identity_blocking's signature. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
When the keyring is unreachable at boot and an identity-transition journal row survives, the durable-B candidate cannot be reconciled this boot. Loading the A-valued identity.key would resume fully-signable A over a possibly-canonical B — the exact split-state the journal exists to prevent. Fail closed to the recovery-blocked posture (ephemeral key, signing disabled) the base migration-marker branch already uses, never RecoveryState::None; a later boot with a reachable keyring reconciles from durable fact. Retires pending_exists's allow(dead_code) now that startup consumes it. Pins an allow on PersistenceOutcome::Committed's storage field (consumed by the not-yet- wired P29/P30 barrier body) to keep the tree clippy -D warnings clean; retired in the sub-commit F zero-allow sweep. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…t (P29/P30-C1) The identity-transition coordinator now drains owner-identity egress between the runtime drain and the durable B dispatch, closing the split-state class where an already-issued lease or durable capability could transmit under A after B commits. Because the coordinator body holds two std-mutex Layer-2 guards it cannot carry across .await, the drain is synchronous: a new `wait_egress_drain_blocking` parks on a Condvar beside the egress registry, notified from both lease Drop impls. Both guards stay held continuously across the barrier — releasing either permits A-runtime resurrection before B commits (the three-phase guard-release shape was ruled UNSAFE). Deadlock-freedom rests on leases taking only the egress mutex; a tree sweep found zero lease-vs-guard sites. The persist stage is now classified against durable fact (P27-C1) under the retained commit fence, after a fsync'd IdentityTransitionPending journal (P28-C1): Committed finishes the in-memory swap and clears the journal; DefinitelyUnchanged compensates the drain; Indeterminate latches fail-closed and leaves the journal for boot reconciliation. `commit_imported_identity` no longer performs the durable persist — it takes the proven storage and does only the in-memory swap, so it runs on the Committed arm alone. Retires the dead_code allows the coordinator now consumes across the egress, persistence, and journal modules. An RAII test guard resets the egress registry on drop so a latched Indeterminate state cannot leak into crate tests that read the indeterminate-gated key accessors without the registry lock. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…s (P26-C1) The coordinator exposed a single pre-commit validity check reaching only the late (fenced) boundary, but P26-C1 mandates TWO gates: an early gate under the held transition lock — before Mesh preflight and drain — so a task superseded while queued on the locks is rejected having done zero disruptive work, and the existing late gate held across the durable commit. A single FnOnce cannot cover both; without the early gate a cancelled phone-recovery would drain runtimes and revoke owner-identity durable capabilities before the late check rejected it, churning the fail-closed side uncompensated. Take two explicit checks (early_validity_check + late_validity_check). Normal import supplies always-Ok for both; pairing supplies two clones of the idempotent task-currency read. Collapse the mesh/non-mesh fork onto one manual workspace_transition acquisition and gate the mesh preflight call so the early gate sits at the same point under both feature configs. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Widen run_identity_transition, import_identity_blocking, and their two drain/compensate helpers to <R: tauri::Runtime>, plus the shared drain_scope_runtimes on the apply_workspace path. Monomorphization makes the existing Wry call sites (import_identity, import_recovered_identity, apply_workspace) compile to identical code — behavior-neutral. This lets the P26-C1 coordinator schedules be driven under mock_builder()'s App<MockRuntime> in the forthcoming §7 fixtures. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…s (P25/P26/P29-C1) Three harness-hermetic §7 fixtures for the C5 identity-transition chunk: - commit_imported_identity closed-world sink scan (P25-C1) — the durable in-memory swap has exactly one call site; a new caller in an existing or new file trips the inventory scan, with two mutation proofs. - signing_keys refusal under the Indeterminate latch (P29-C1) — the checked accessor's integration point, with a Live control read proving transparency. - early-gate zero-disruptive-work schedule (P26-C1) — drives the real coordinator through a superseded early_validity_check and asserts the egress state, persistence generation, and active scope are all untouched; the late gate asserts unreachable. The no-scope-commit and None->Some activation-race §7 schedules are not included here: both require the coordinator to reach a durable commit, which persists through the real process-global keyring and the real app_data_dir under mock_builder() (empty identifier). Routing that hermetically is a separate question raised to Paul. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…dinator The commit path in import_identity_blocking resolved app_data_dir() and SecretStore::shared() internally, so a mock-driven fixture that drives the coordinator to a durable commit would write under the real ~/Library/Application Support and bind the process-global keychain. Hoist both durable-persist inputs — data_dir: PathBuf and store: &impl IdentityKeyStore — to the run_identity_transition boundary; both production callers (import_identity, import_recovered_identity) pass the real values they resolved before. This is dependency injection of genuine persist inputs (the downstream persist_imported_identity/classifier already take exactly these), not a test-only abstraction: it lets §7's no-scope and None->Some activation-race fixtures drive a real commit hermetically against a tempdir and a fake store. Zero logic change; monomorphized call sites are behavior-neutral. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… (P26-C1) Adds the three coordinator-driven §7 recovery fixtures that require a real durable commit, now hermetic via the injected persist seams: - no-active-scope recovery: acquires workspace_transition, selects the no-scope branch from the under-guard snapshot, commits B through a fake reachable store + tempdir, and bumps the persistence and scope generations exactly once each; - None->Some activation race, both resolved orders — recovery-wins (commits under a true no-scope snapshot, activation then applies against B) and activation-wins (the recovery's under-guard snapshot observes the pre-committed scope and runs the full active-scope drain/commit/clear). A Send+Sync SyncKeyringStore backs the commits; the RefCell FakeIdentityStore is !Sync and cannot cross the coordinator spawn_blocking. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Every #[allow(dead_code)] tagged "remove when C5 lands" now either gains its production consumer or is retired, per the zero-allow sweep. The test-only accessors (current_identity_persistence_generation, identity_persistence_state, await_egress_drain, read_pending, OwnerIdentityCapability::generation, live_durable_capability_count) move to #[cfg(test)] rather than carrying a speculative allow. The unread ManagedAgentEgressLease.generation field and getter are deleted: the coordinator drain awaits via in_flight, never this field, so the value was dead in every config. Production-consumed symbols (Draining, revoke_durable_capabilities_before, OwnerIdentityEgressLease.generation) simply drop the allow. The EgressLease variant payloads keep one documented allow: they are RAII witnesses held for Drop, never read out. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Fold current origin/main into the cross-workspace agent library branch. Base was re-taken from 934f332 after main rewound past the prior staged base (203735f): the TTS-playback merge was reverted and #6271 (buzz-dev-mcp ~ expansion) and #6261 (buzz-acp workspace-scan) landed. The 12 Rust conflict files were byte-identical between the two bases, so every resolution replayed 1:1. Non-conflict main additions (#6271/#6261, identity-persistence coordinator, owner-identity egress) merge cleanly. Re-thread #6003 workspace-apply staleness guard, a one-sided main addition the prior resolution dropped: app_state fields + init, the next_apply_generation/assert_current_apply_generation/begin_workspace_apply helpers, WORKSPACE_APPLY_SUPERSEDED, and main's tests, with the apply lock transferred into HEAD's restructured fire-and-forget restore spawns so the guard survives the reshaped restore path rather than pasting main's now-incompatible spawn. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The merge unites main's identity-archive tests, which transitively admit an owner-identity egress lease through submit_event/signing_keys, with this branch's owner_identity_egress tests, which latch the process-global registry into Draining/Indeterminate. Neither parent had both test sets. The archive tests serialized only on the rate-limit TEST_SERIAL, not the EGRESS_REGISTRY_TEST_LOCK the egress tests use, so a concurrent egress test's drain window leaked in and refused the signer — a reproducible "owner-identity egress is draining" / empty-snapshot failure under the full parallel lib run, passing in isolation. Hold EGRESS_REGISTRY_TEST_LOCK and reset the registry in the three egress-touching archive tests, matching the module's own guard idiom and the TEST_SERIAL-first lock order of the egress admission tests. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Bind every direct owner-key artifact read to its admitted egress generation so an identity transition cannot stamp an A-derived artifact as B-current. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Serialize NIP-49 backup creation with identity transitions before acquiring an egress lease, preventing the drain barrier's lease-versus-mutation deadlock. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96
force-pushed
the
duncan/workspace-scoped-agent-store
branch
from
August 19, 2026 22:31
a576e06 to
5d4f2c0
Compare
Discover every owner-identity egress candidate structurally and require its explicit classification, so a newly introduced artifact producer cannot evade C2 ordering checks. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ed-agent-store Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR partitions the agent definition store by workspace, closing a two-directional leak where definitions (managed agents, teams, global config) and their runtime connections were shared across relays.
Previously a single unscoped store lived at
agents/and everyapply_workspaceevent-synced all records to the newly selected relay, while the runtime reconciler fanned agents into every configured community. After an identity switch, in-flight old-owner inbound events could write into the new owner's store.What this PR does
WorkspaceAgentScope { scope_id, relay_url, owner_pubkey, definitions_dir, generation }as the single scope authority;scope_idis byte-identical to the retention DB's sha256 derivation via a shared helperapplyCommunity()anduseCommunityInit.tsAdoptedLegacy | LegacyClaimedByOther | FreshNoLegacymanifest, atomic rename, versioned_readymarker v1) that uses the existingretention.dbclaim as the canonical ownership ledger so retention and definition adoption can never diverge_readymarker: an unversioned or v0 marker forces re-run throughrun_pre_ready_family(retention migration + persona backfill) before advancing to v1, so scopes created by earlier defective pipeline iterations are repaired on next activation. Marker written via temp+rename (atomic)scope_for_arrivalto match(relay, owner)so in-flight old-owner inbound events cannot land in the new owner's store after an identity switchworkspace_transitionthrough runtime installation so switches cannot race a concurrent client start. Serve-mode runtimes are machine-level and never block a switch. The journaled Mesh recipe is tracked as a follow-up below.start_on_app_launchagents on every activation; enforce relay-match reuse onensure_relay_mesh_for_recordwith a fail-closed preflight error when a serve-mode runtime belongs to another relaycommunitiesparameter fromreconcile_managed_agent_runtimes; both frontend callers now invoke a parameterless command whose relay is derived server-side from the active scope, making cross-scope fan-out unrepresentableset_global_agent_config) captures scope at entry; Phase 1 validates generation inside the store lock before writing; Phase 2 per-agent restart validates under lock before stop — all I/O targets the captureddefinitions_dirviastart_local_agent_pairs_with_preflight_atconfirm_agent_snapshot_import,confirm_team_snapshot_import) capture owner keys at entry, verify pubkey against captured scope, and thread them through all mint/retention/engram phases. Re-verify under store lock before Phase 3a write. All outbound profile/memory publication usescaptured_scope.relay_urlidentity_mutationbefore egress admission, eliminating the inverse lease-versus-mutation deadlock; a mock-runtime schedule proves a backup queued before the drain holds no lease until transition exitcompensate_draintakes the caller's already-heldmanaged_agent_runtime_transitionguard by value, re-acquires only the store lock, validates captured scope generation, then restores journal entries viastart_pair_under_held_locks— closes the drop-then-compensate interleave window without recursive locking or an AtomicBool gatemigrate_persona_provider_to_runtimeruns as step 0 ofrun_scoped_migrations(returnsResult, propagates errors);migrate_agent_keys_to_dev_serviceruns inrun_pre_ready_family(debug non-test builds only, returnsResult). Pre-scope calls inrun_boot_migrations_innerremoved.workspace-degradedTauri event wired touseNestNotifications.ts(toast.error with cleanup + behavioral test); restore spawn emits the event on failure;spawn_event_syncreturn type corrected to()(fire-and-forget; dispatch failure during runtime shutdown has no toast surface)Deferred follow-up (tracked here, no separate issue)
Journaled Mesh recipe: client-mode Mesh runtimes have no persisted restart recipe (restore only knows Serve mode). The correct end-state is to include the Mesh client in the drain journal with ownership and restart data, coordinate drain via
rearm_lock → mesh_llm_runtimein the drain stage after prepare, and compensate synchronously on failure. This requires building the restart recipe for consumer-mode clients first. Option A (fail-closed switch while a client runtime is active, with user-accessible stop command) is the interim behavior for this PR.