🤖 refactor: coordinate compaction and durable continuations - #4121
🤖 refactor: coordinate compaction and durable continuations#4121ThomasK33 wants to merge 21 commits into
Conversation
Compaction work can finish after its originating turn has been canceled or replaced. Give compaction observations and durable continuation handoffs explicit coordinator ownership so late work cannot clear a replacement's state, replay a continuation after Stop, or report a handoff that never reached durable acceptance. - Track semantic compaction intent, observation stages, and handoff tokens in the turn coordinator. A continuation preserves its ownership across its own admission; unrelated manual or edited turns retire it. - Keep original eager summary jobs and journal cleanup physically leased through reset and shutdown. Reuse the existing journal queue, and guard completion callbacks against retired intent. - Guard summary updates and heartbeat-boundary cleanup under the existing history lock and at final publication. Serialize pending-state writes and unlinks, and detach rollback snapshots before awaiting I/O. - Count a durable continuation as accepted only when its acceptance callback runs. Preserve ordinary continuation queue priority and the existing manual-input vetoes for optional and goal continuations. - 2,039 backend tests passed across session/compaction, history, workspace/task/container, and CLI suites. - 15 UI tests passed across compaction, interruption, and send modes; one existing compaction test remains skipped. - Deterministic regressions cover held preparation, summary generation, journal clearing, history reads/writes, Stop, replacement, rollback, and successful sends that never reach acceptance. - The automatic-compaction UI fixture now explicitly configures its mock provider and compaction model. The original fixture timed out on unchanged main without those prerequisites; the corrected fixture passes on both main and this branch. - The separate credential-backed context-limit UI suite could not run locally because `OPENAI_API_KEY` is unavailable. The mock-provider compaction suite covers context-limit recovery. This changes ownership across continuous, legacy, and heartbeat compaction and startup replay. Cancellation ordering, manual-turn priority, and durable cleanup are the primary regression risks; the held-I/O and UI tests exercise those boundaries. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: I8ff308b1367d223a86fe3914da56c8e928f334df
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
Security findingsAdvisory findings (2)
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@codex review Please review the compaction and continuation ownership changes, particularly Stop versus replacement cleanup, durable acceptance, and shutdown joining original asynchronous work. Local backend and mock-provider UI validation is complete. Generated with |
This comment has been minimized.
This comment has been minimized.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 44d48d3b8d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Record supersession at the session failure source so replacement after durable acceptance cannot restart the predecessor's recovery path. Genuine startup failures retain their error handling even when a replacement arrives later. Reject new dispatch claims after Stop and allow a separate guarded cleanup claim to remove the abandoned durable follow-up without sending it or changing a replacement's history. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: Ia08d923a50745a76ccc2bab5235820ebf5b1e90e
|
Addressed both findings:
The coherent session/compaction suite passes all 772 tests; workspace/task/container suites pass 1,161 tests and full static checks pass. Generated with |
|
@codex review Both P1 findings are fixed in Generated with |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4af8d68d3b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Restore the captured pending state and publish boundary deletion synchronously with the guarded history commit. A replacement admitted during later cleanup can no longer observe a deleted boundary with stale attachments. Join the ordered persistence afterward and preserve committed-write facts through cleanup errors. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: If698c58a4512d3a41536e56f413a003e5f0dc82f
|
Addressed The regression gates the real post-rename filesystem cleanup and admits a replacement while it is held. It verifies that the replacement sees restored attachments and that the renderer deletion has already been published. Additional cases cover cleanup errors after commit, throwing observers, guard vetoes, archived/missing targets, and late persistence that must not overwrite a successor's snapshot. Validation passes: 777 session/compaction tests, all 102 real HistoryService tests, 1,161 workspace/task/container tests, and full static checks. Generated with |
|
@codex review The heartbeat rollback commit/publication race is fixed and covered by a real post-rename cleanup gate. Please review the synchronous commit observer, state restoration/publication ordering, and postcommit error handling. The finding has a detailed reply and is resolved. Generated with |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e5bed7367
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Keep rollback scoped to the compaction handoff that persisted the rows. Stop retains abandoned cleanup ownership; a superseded handoff cannot invalidate a successor. Cover held append before and after commit, successor preservation, and restart recovery. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: I0b812c3898c4c0c9e54de4f41dde21e8d35eb4de
|
Addressed A four-case regression gates the real append both before and after persistence, with and without a manual successor. It reproduced all four failures before the fix and now verifies cleanup, successor row/token/summary preservation, and no canceled continuation on restart. The 781-test session/compaction group and full static checks pass. Generated with |
|
@codex review The Stop-during-append cleanup ownership finding is fixed and covered on both sides of the persistence boundary, including manual successor and restart recovery cases. Please review the captured handoff guard and retained abandoned cleanup behavior. The finding has a detailed reply and is resolved. Generated with |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 75808743a0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Acknowledge the compaction handoff at the irrevocable row frontier, preserve actual startup errors, and keep abandoned cleanup joined through shutdown. Handle rejected goal work with captured cleanup ownership without removing an accepted continuation's recovery marker. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: I44b6643d5f3c64d8e9bf1f1b8235dce240ffe0b5
|
Addressed both findings and adjacent rejection windows:
The regression matrix covers held goal sync, synchronous row observers, failed rollback, cleanup before and after claim, guarded rewrites through shutdown/disposal, and rejected goal admission/update/publication. Rejection cleanup uses the captured summary and durable receipt, preserving successor state and original error reporting. Independent review reran both additional rejection reproductions red-to-green. The coherent session/compaction group passes 800 tests and full static checks pass. Generated with |
|
@codex review Both durable-handoff and shutdown-cleanup P1s are fixed. The receipt now precedes all post-durability observers/awaits, abandoned cleanup remains joined through shutdown, and rejected goal work follows the same guarded cleanup obligation without masking the original error or clearing an accepted row's marker. Both findings have a detailed reply and deterministic regression coverage. Please review the durable frontier and exact cleanup ownership paths. Generated with |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 83ed03e3fb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Retry an initial failed summary read once under the retained abandoned owner, update only the captured pending handoff from the locked current row, and reset abandonment when a replacement context advances the epoch. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: Ifb99c5fbc1fca3d7c91b98ec29794a96afa00928
|
Addressed all three findings:
Twelve new regressions cover targeted/untargeted thrown and Result.Err reads, persistent failure bounds, successor preservation, late summary updates, changed pending requests/IDs/sequences, and actual replacement dispatch. Independent review reran all twelve successfully. The coherent session/compaction/history group passes 914 tests, including all 102 real HistoryService tests; full static checks pass. Generated with |
|
@codex review The three cleanup-identity findings are fixed and covered by twelve regressions. Please review the bounded same-owner reread, locked-row transform that preserves unrelated fields, and replacement epoch abandonment reset. Each finding has a detailed reply and is resolved. Local coherent validation and static checks pass. Generated with |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e5a0b0c4b5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Retain failed canceled cleanup under its exact owner across recovery, disposal, and lazy session recreation. Retry through bounded lifecycle operations and report unresolved teardown after releasing resources. Temporary admission holds preserve cancellation; committed context replacement explicitly starts a new intent. Restore local heartbeat state when another backend already deleted the exact boundary, without duplicate publication. Integrate main's merged streaming replay fix (#4123) while preserving both coordinator regression suites. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: I3f7ce21ee0d9b837e8fa79354af6a0244c4eb74f
|
@codex review Addressed findings 3955121897, 3955121906, 3955121915, and security finding 3955164918. Please review the complete change, especially retry delivery after Stop, failed-publication retry ownership, full-clear publication fencing, and source-less handoff generations. The preceding reply explains the fixes and corrected backoff regression coverage. Generated with |
Preserve both compaction publication ownership and main's pinned model capability. Include the merged remembered-model and chat-hydration changes. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: I9322206facb3d4cf6aa8076a8c37219f5939533f
|
@codex review Integrated current main through d8d368d to resolve the import conflict. Both compaction publication ownership and main's AIService capability are retained. The merge also preserves manual-only remembered settings, pinned summary routing, and chat hydration from #3968, #4124, and #4125. Please review the complete combined change; the preceding response covers all four addressed findings, and the PR description records validation of the merged revision. Generated with |
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d67b2563f5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Invalidate captured compaction publications under the same history lock as reset and destructive context writes. Preserve rejected and non-destructive operations, and refuse unfenced writes when generation publication fails. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: I57ef548fbdb7ab24962f29d00a986cfcfa15c0bd
|
Addressed 3955716369 at the actual history-write boundary. Explicit reset rows, including reset-containing rollover batches, advance the shared publication generation under the history lock after admission checks and before the write. Full deletions and partial trims that remove active provider context advance it before their destructive rewrite. A failed generation publication refuses the context mutation; declined operations, no-op trims, and trims of inactive history preserve the existing journal. Ordinary compaction boundary rotation does not invalidate its own publication. Deterministic tests hold actual foreign heartbeat and legacy publishers across reset, replacement, and active-context trimming with no existing Stop. The tests also cover generation-write failures, batch admission before invalidation, and preservation of journal bytes for rejected or non-destructive operations. Existing initial and retry rollover tests verify the intended continuation still starts with its pinned request. Final validation is recorded in the PR description. Generated with |
|
@codex review Addressed P1 3955716369 by advancing the shared generation inside reset and destructive history writes, including reset-containing batches and active-context prefix trims. Please review the full change, particularly admission-before-invalidation ordering, failed generation writes, and preservation of intentional rollover continuation and inactive-history journals. The preceding reply explains the fix; the PR description records final validation. Generated with |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4804894622
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Invalidate foreign compaction publications inside edit truncation writes. Tie provisional attachments to their intended durable boundary and clean exact owned writes after cancellation, preserving newer preparations and valid prior state. Signed-off-by: Thomas Kosiewski <tk@coder.com> --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable --> Change-Id: I5748fc35360ba556cf5f444bbc0433663b175144
|
Addressed both findings:
Tests cover restart before cleanup, cleanup failure, lost write acknowledgements, same-content successor writes, prior committed state, stale loading, and actual attachment delivery after successor preparation. Final validation is recorded in the PR description. Generated with |
|
@codex review Please review the updated PR, including the edit-history generation fence and provisional attachment ownership fixes for 3956050520 and 3956050532. Both findings have replies and regression coverage. Final validation is in the PR description. Generated with |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e95ef2c9b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| fingerprint(current) === snapshotFingerprint | ||
| ); | ||
| }, | ||
| publication: { generation: staged.publicationGeneration }, |
There was a problem hiding this comment.
With multiple backends enabled, this publication fence protects only the inner boundary commit: withContinuousPendingState() writes post-compaction.json before this call and, when applied is false, later restores or deletes that file without checking the write receipt or shared publication. If backend B invalidates this generation and publishes a successor pending state before backend A's wrapper reaches its finally, A can overwrite or unlink B's state, causing the successor's file, skill, and read-path attachments to be lost. Fresh evidence beyond the replied provisional-state finding is that the continuous path still bypasses the new receipt-based cleanup used by heartbeat and legacy completion; pass the publication through the wrapper and condition cleanup on exact ownership.
AGENTS.md reference: AGENTS.md:L112-L112
Useful? React with 👍 / 👎.
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
|
Closing at the author's request. This PR grew beyond a reviewable unit and will be replaced by smaller, independently safe PRs, with tests accompanying each behavior change. Planned slices: physical compaction lifetime; coordinator extraction; local pending-state ownership; correlated continuation handoff; journal publication; destructive-history fencing; pending attachment persistence; durable Stop/replacement. Dependent changes will use shallow stacks and land bottom-up. Repeated substantive review rounds will trigger a scope review rather than continued accumulation. The branch and unfinished local work are preserved as reference. This closure does not indicate that the remaining review finding was fixed or that the PR was merge-ready. Generated with |
Reset can clear the active eager-compaction job while its preparation or summary request is still running. Session shutdown then loses track of that detached work and can finish too early. Acquire the existing coordinator execution lease when the eager job starts, and release it in that same promise's `finally`. Semantic reset and normal turn admission retain their existing behavior. This is the first small replacement for closed coder#4121. It only fixes physical lifetime ownership; coordinator extraction and persistence hardening are separate changes. Validation covers held preparation and summary work across completion, rejection, reset, shutdown, and replacement by another eager job. The tests retain real coordinator leases and HistoryService storage. Risk: shutdown now correctly waits for eager work that ignores cancellation until it settles. Existing cancellation and semantic idle behavior are preserved. --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable -->
Compaction observation state is split across several AgentSession flags and a separate waiter list. Centralize those transitions in the existing turn coordinator, with a token identifying the observation that may update or finish them. This behavior-preserving layer sits above coder#4129. AgentSession retains its physical promise, and the pending window remains open through continuation dispatch and cleanup. Admission, terminal ordering, persistence and shutdown timing stay the same. Stale tokens cannot release a replacement observation's state or waiters. This replaces the coordinator-extraction portion of closed coder#4121. Local pending-state ownership, continuation handoff changes and durable persistence protocols remain separate PRs. Validation: 635 targeted tests and `make static-check` passed. Behavioral regressions cover stale-token isolation, waits through failed dispatch/cleanup, retained pending state during shutdown/disposal, summary targeting and A's eager-job leases. Independent local review found no issues. Risk is concentrated in compaction waiting and queue release. Existing session promise ordering is retained to keep this extraction independently reviewable. --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable -->
Compaction journal writes and history folding use different serialization boundaries. A delayed fallback or cleanup can therefore operate on a journal that has already been replaced, and a failure after a committed history rename can still look like an uncommitted compaction to its caller. Serialize journal publication and folding with the existing cross-process history lock. Compare the captured generation and exact journal, and publish ownership/commit receipts synchronously with the final rename. Keep a committed boundary's pending-state success latch set even if later cleanup fails. Explicit reset remains a separate operation with its established local queue, clearing and invalidation behavior, including an unread startup journal. It does not acquire the shared history lock, so lock contention cannot abandon reset unlink. The advisory existence probe also uses the local queue; actual journal reads and mutations keep the shared lock. Asynchronous owned cleanup uses an exact receipt; completing an apply does not perform a second global clear. Invalid, stale, source-mismatched and durably applied journals preserve their recovery outcome when cleanup fails, leaving exact cleanup retryable. These boundaries let the change ship independently of later destructive-history fencing, attachment-persistence and durable Stop protocols. This is the journal-publication slice replacing closed coder#4121 and starts a separate persistence stack. Validation: the broader implementation suite passed 370 targeted tests; the final cleanup changes passed all 168 affected tests and `make static-check`. Regressions cover interrupted startup recovery, reset under a held history lock, failed cleanup and later recovery, successor preservation, queued fallback/reset ordering, cross-instance publication, commit receipts before cleanup, and no rollback or duplicate folding after a committed boundary. Independent local review approved the implementation and review fixes. Nix formatting was skipped because Nix is unavailable. Risk centers on journal recovery and lock ordering. Under-lock operations avoid re-entering the journal queue; tests use real HistoryService storage and deterministic barriers. The ordered file protocol does not add a new power-loss durability guarantee. --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable -->
Queued compaction cleanup can resume after a replacement turn and overwrite newer summary metadata or delete its heartbeat boundary. Make these two cleanup operations conditional under the existing history lock: match the captured summary ID, sequence and pending request, transform the current row, and recheck local ownership immediately before rename. Skipped heartbeat cleanup restores no pending state and emits no deletion. Ordinary history update/delete APIs and serialized cleanup waiting remain unchanged. This small prerequisite above coder#4134 replaces part of closed coder#4121. Correlated continuation handoff will be a separate PR; this change does not enable overlapping retirement or add durable cancellation. Validation: the original broader run passed 303 tests; the final adapted layer passes all 176 affected tests and full static checks. Coverage includes exact cleanup, 10 corrupt-sequence cases, duplicate-identity skips, same-ID/different-sequence controls, and session races while cleanup is queued or its committed result is delayed. Invalid or ambiguous identity evidence returns a recoverable skip without permitting deletion by ID alone. Independent review approved the adaptation; coordinator review approved the final uniqueness guard. Risk: an incorrect match could clear the wrong pending follow-up or skip a valid cleanup. Tests cover current-owner success, stale-owner rejection, unrelated metadata preservation, staging failure, and preserved history sequence/archive behavior. --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable -->
Introduce an inactive cancellation state core for the agent-loop refactor. It keeps cancellation nonce and retry ownership across failed persistence, distinguishes witnessed replacement from unfinished deletion, and prevents stale narrowing or retirement from superseding newer work. Storage is an injected contract. A later adapter must enforce shared-lock nonce/frontier comparison, verify replacement witnesses, and repair malformed storage safely. No runtime caller uses this core yet; Stop and recovery behavior are unchanged by this PR. This independent H1 prerequisite replaces part of closed coder#4121. It does not depend on the pending-attachment protocol, so it can be reviewed separately without unrelated ancestry. Validation: 48 behavioral tests pass, including narrowing versus retirement, fallback read rejection versus newer Stop/retry, foreign narrowing after failed witnessed deletion, and reordered reads and repairs. A newer accepted read supersedes older results; a merely pending successor cannot hide valid cancellation or repair errors. Final full static checks and independent review pass. Nix formatting was skipped because Nix is unavailable. Diff: 318 production lines and 727 test lines in two new files. Risk is limited while inactive. Injected-adapter tests prove state-machine invariants, not filesystem durability or cross-process compare-and-swap; the real adapter and complete runtime activation remain separate deliveries. --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable -->
) A compactor in another backend can retain a snapshot after reset, rollover, truncation or context-budget rejection. Advance the journal publication generation under the same history lock before those mutations, so stale work cannot publish or fold into the changed history. This is the reset/truncation/context-budget fencing layer above coder#4133, split from closed coder#4121. Refused operations, no-ops, sealed-prefix or targeted display-only cuts, and ordinary compaction preserve publication ownership. Cleanup deletions through deleteMessage/deleteMessages and their private helper are a separate F2 follow-up; this PR does not claim to fence every history mutation. Context-budget rejection also retires publication when it removes eligible provider context. Tail cuts recognize removal of a durable reset or compaction boundary even when it contains no replayable text. Prefix cuts use the raw-reset-aware active suffix, preserving generation and usage metadata for rows sealed by malformed reset evidence. Validation: 347 tests pass across seven history/compaction suites with default limits after rebasing onto the journal review fixes. They cover foreign stale publication/fold rejection, fresh publication success, failure before destructive writes, lock ordering, context-budget rejection, removal of empty boundaries, and raw reset privacy floors. Full static checks and independent local review pass. Risk: a later history-write failure can conservatively leave the generation advanced, discarding an in-flight compaction while retaining history. Durable Stop and attachment persistence remain separate layers. --- _Generated with `xum` • Model: `unavailable` • Thinking: `unavailable` • Cost: `$unavailable`_ <!-- mux-attribution: model=unavailable thinking=unavailable costs=unavailable -->
1 participant
Summary
Compaction can finish after its turn has been stopped, cleared, or replaced. Give observations and continuation handoffs explicit coordinator ownership, and persist Stop independently of transcript cleanup. Late work cannot clear a replacement's state or revive canceled work after restart. Temporary admission holds resume recovery when released.
Implementation
The coordinator owns compaction intent, observation stages, and handoff tokens. Semantic retirement wakes waiters without joining obsolete physical work; summary jobs and cleanup retain execution leases through shutdown. Failed cleanup remains bounded and retryable outside the live-session registry.
Continuation admission and cleanup validate exact ownership under the shared history write lock. Summary handoffs check row identity, pending payload, permitted tail, and cancellation; source-less handoffs carry the original stream's durable publication generation. Legacy, heartbeat, and continuous summary publication and partial cleanup use the same generation fence. Pending-context preparation, malformed-state cleanup, and failed-publication rollback carry that generation through their queued filesystem writes; a rejected old summary cannot overwrite or delete a successor's context.
A versioned atomic cancellation sidecar survives transcript failures. Each explicit Stop gets a distinct nonce and advances the durable publication generation before publishing. Failed writes retain their predecessor and retry only that frontier; concurrent replacement reads share a retry, and obsolete retries preserve newer journals. Full clear retains cancellation until actual explicit replacement, preventing late summaries from repopulating cleared work. Reset rows and reset-containing batches, full deletion, and trims that remove active provider context advance the shared generation inside their actual write lock after admission checks and before mutation. Failed generation publication refuses the context write; no-op, rejected, and inactive-history-only trims preserve the journal.
Acceptance occurs at the irrevocable row frontier before goal synchronization or observers. Every manual append branch and explicit Retry compares captured cancellation identity under the accepting write lock. Internal handoff snapshots and their user receipt commit atomically, including with token budgets disabled. Skipped writes cannot acknowledge acceptance; genuine I/O failures remain errors.
Exact replacement witnesses in active or archived history retire matching cancellation. Failed unlink is ancillary debt and does not block accepted work. Live dispatch and cleanup reconcile these witnesses without erasing a newer Stop. Automatic retries cannot witness replacement and recheck unresolved Stop before scheduling, after backoff, and before provider entry.
Malformed cancellation self-heals under the history lock: quarantine bytes, invalidate old generations and journals, and sanitize pending follow-ups before removing the corrupt fence. Failures preserve protection and retryability. Caller-scoped journal reads and identity-scoped cleanup cannot adopt or delete a successor's journal; repair bypasses the local journal queue to avoid self-deadlock.
Preserve main's token-budget rollovers, pinned request assembly, raw reset privacy boundaries, and append provenance. Retain deferred Bash-monitor wakes, cancelable wake withdrawal, semantic pending-compaction waiters, and both durable Stop obligations from 🤖 fix: defer bash-monitor wakes until the active turn ends #4114. Preserve current main's manual-only remembered settings, pinned summary model routing, and chat hydration from feat: remember workspace model and mode on send #3968, 🤖 fix: enable GPT-6 Astra Pro mode through Coder #4124, and 🤖 feat: show chat history loading above the composer #4125. The producer-to-frontend replay fix from 🤖 fix: restore live chat streaming after reconnect #4123 remains covered.
Active and archived edit truncations also advance the durable generation before removing provider context. Provisional legacy and heartbeat attachment state names its intended durable boundary, so a crash cannot turn an uncommitted summary into usable attachment state. Failed or skipped publication cleans only its unique file write under the shared lock; prior bytes are restored only while the original publication remains current. Separate preparation identity protects live memory from stale loads, denied writes, and cleanup racing a newer queued preparation.
Validation
make static-checkand a freshmake build-mainpass. The compaction UI suite passes 9 tests with 1 existing skipped test.Deterministic real-HistoryService regressions cover foreign backends, held I/O, failed publication/cleanup/rollback, active and archived witnesses, malformed-state repair, exact durable acceptance, and fresh-service recovery. Retry tests advance the actual RetryManager clock and join delivery, with positive controls; they do not infer safety before backoff expires. Held-generation tests check that retired preparations and compaction publishers preserve successor context, partial responses, and journals.
Risks
This changes ownership across continuous, legacy, and heartbeat compaction and startup recovery. Cancellation ordering, manual-turn priority, and durable cleanup are the main regression risks. Storage errors remain visible and retryable; failure of both transcript cleanup and cancellation publication cannot provide crash durability. Older builds honor successful legacy pending-marker cleanup but cannot read the new fallback sidecar. If cleanup remains impossible, downgrading loses the added restart protection; the parent build already resumes that failed-cleanup state after its own restart.
Pains
Several tests previously asserted state before scheduled work actually ran. These now use explicit completion barriers or the injected retry clock. Integration required preserving concurrent token-budget and Bash-wake changes on main. Multi-backend races require both local identity checks and durable comparisons at the actual write boundary.
Generated with
xum• Model:unavailable• Thinking:unavailable• Cost:$unavailable