Skip to content

fix(ai): keep a chat turn's Durable Object alive with a heartbeat alarm - #905

Draft
Makisuo wants to merge 1 commit into
inv/05-tool-failures-and-partialsfrom
inv/06-turn-heartbeat
Draft

Makisuo wants to merge 1 commit into
inv/05-tool-failures-and-partialsfrom
inv/06-turn-heartbeat

Conversation

@Makisuo

@Makisuo Makisuo commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Stack 6 of 6, draft: not yet verified end to end. Based on #903.

Why

A plain fetch() never keeps a Durable Object in memory, even while the response body streams, and an object with no incoming request or event for 70–140 seconds is evicted (Cloudflare docs). A chat turn survives because the open page holds a subscription. An autonomous investigation that nobody is watching has nothing holding it. After #898 that is every alert-triggered investigation.

Locally, an investigation started from the hub with its page never opened went silent 2m17s in, right after a batch of tool results. It raised no error, the two-minute stream idle timeout from #903 never fired, and there was no close-out. It sat until the 15-minute stale sweep marked it failed. A timer that never fires suggests the fiber itself was gone, which fits eviction.

What changes

  • While a turn holds the slot, the object re-arms its alarm every 30 seconds. The alarm is the event that prevents eviction.
  • If the alarm lands on an activation that holds the claim but is not running the turn, the object was evicted anyway (a deploy). It then releases the slot with a terminal event instead of waiting for the watchdog.

Not verified

The confirming run did not complete. The first attempt was invalid because closing the Browser pane stopped the dev stack underneath the run. Two restarts then hung before planning, while the local Docker Postgres stopped answering queries. Before merging, run one unobserved investigation on a preview or locally and confirm it reaches diagnosed or inconclusive, and confirm alarms fire under alchemy dev.

Unit-tested against the real SQLite fake: the alarm is armed on beginTurn, re-armed while the turn runs, not re-armed after it ends, and an orphaned slot is released on a fresh activation. Typechecks pass on apps/ai, apps/api and apps/alerting.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

An outbound fetch never keeps a Durable Object in memory, even while the
response body streams, and an object with no incoming request or event
for 70-140 seconds is evicted. A chat turn survives because the open
page holds a subscription. An autonomous investigation nobody is
watching does not: running end to end locally, an unobserved pass went
silent about two minutes in, with no error and no close-out, and sat
until the 15-minute stale sweep marked it failed. Every alert-triggered
investigation is unobserved.

While a turn holds the slot, the object re-arms its alarm every 30
seconds; the alarm is the event that prevents eviction. When the alarm
lands on an activation that holds the claim but is not running the turn
— the object was evicted anyway, by a deploy — it releases the slot
with a terminal event instead of waiting for the watchdog.
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant