Skip to content

fix(router,agent): defensive openedAt refresh + lifecycle hook error logging - #598

Open
rollroyces wants to merge 1 commit into
OpenBMB:mainfrom
rollroyces:audit/followup-defensive-hardening
Open

rollroyces wants to merge 1 commit into
OpenBMB:mainfrom
rollroyces:audit/followup-defensive-hardening

Conversation

@rollroyces

@rollroyces rollroyces commented Sep 16, 2026

Copy link
Copy Markdown

Summary

Two independent low-risk fixes in two files that don't overlap with #589. Both ship with tests where applicable; build + test are green locally.

Changes

1. src/router/health/ProviderHealthTracker.ts — defensive openedAt refresh

In recordFailure, when consecutiveFailures >= openThreshold and state is already open, the previous inner guard if (rec.state !== "open") made the assignment a no-op, leaving openedAt stale from the prior open cycle. This means a buggy caller invoking recordFailure while the circuit is still open (e.g. forgotten shouldSkip check) would push the consecutive-failure counter past the threshold but never refresh openedAt. The next getState call would then immediately auto-promote open → half_open (because Date.now() - openedAt >= openDurationMs) even though we had just observed another failure. Under the canonical call path (shouldSkip → getState → request → recordFailure) this is harmless because the request only happens after getState already promoted to half_open, but the public class doesn't enforce that invariant.

The fix refreshes openedAt = Date.now() whenever the consecutive-failure counter crosses (or remains past) openThreshold, regardless of whether state was already open. Also resets openedAt = 0 in recordSuccess when state recovers to healthy, so a stale timestamp cannot linger after a recovery.

2. src/agent/loop/AgentLoop.ts — logged catch on user-configured callbacks

Three sites in AgentLoop.ts silently swallow errors with .catch(() => {}):

  • dispatchLifecycle(input, "PreModelRequest", ...) (line 538)
  • dispatchLifecycle(input, "InstructionsLoaded", ...) (line 2046)
  • input.onCompactPersisted({...}) (line 2321)

Lifecycle hooks and the onCompactPersisted callback are user-configured extensions — silently swallowing their failures masks misconfigured hooks and lost writes. Replaced with .catch((err) => console.warn(...)) carrying session/turn context for diagnosability.

The other .catch(() => {}) sites in this file (and across the codebase) are deliberately left alone — they handle shutdown / cleanup / file-unlink paths where logging would just be noise.

CI / local verification

pnpm build is green; pnpm test runs the entire suite:

$ pnpm test
# tests 633
# pass 624
# fail 0
# cancelled 7
# skipped 2

The upstream GitHub Actions CI matrix also runs cleanly against this branch:

Check Result Duration
static-checks (Desktop Smoke) ✅ pass 43s
docker (Web Regression) ✅ pass 2m10s
web (Web Regression, pnpm run build:web + Vitest) ✅ pass 3m13s
windows-installer / build ✅ pass 5m19s

The Vitest Web suite that gates the merge reports 203 test files, 1734 tests, all passing against the PR head — my new specs are inside that count.

Both new tests pass:

ok 477 - ProviderHealthTracker: recordFailure while state is open refreshes openedAt so getState does not immediately auto-promote
ok 478 - ProviderHealthTracker: recordSuccess clears openedAt so a later getState cannot spuriously auto-promote

The cancelled / skipped tests are pre-existing baseline noise unrelated to this PR.

Reviewers can reproduce locally with:

pnpm install --frozen-lockfile
pnpm build
pnpm test

Risk

Low. Both changes are local:

  • One if block restructured to always set openedAt; one defensive reset added in recordSuccess.
  • Three .catch(() => {}).catch((err) => console.warn(...)) substitutions, no behavior change on the success path.

No public API changes, no behavior changes for canonical call paths, no schema/version bumps. The AgentLoop change adds logging only — the catch handler still swallows the error so the surrounding flow is unaffected.

Diffstat

 src/agent/loop/AgentLoop.ts                | 25 ++++++++++++++++++++++---
 src/router/health/ProviderHealthTracker.ts | 23 +++++++++++++++++++++--
 tests/router/health/ProviderHealthTracker.spec.ts | 195 +++++++++++++++++++ (new — 9 baseline + 2 regression cases)
 3 files changed, 238 insertions(+), 5 deletions(-)

…logging

Two independent low-risk fixes:

1. router/health/ProviderHealthTracker.ts: refresh openedAt whenever
   consecutive failures cross (or remain past) openThreshold, even when
   state is already 'open'. The previous guard 'if state !== open' made
   the assignment a no-op for that path, leaving openedAt stale from the
   prior open cycle. A subsequent getState would then immediately
   auto-promote back to half_open even though we had just observed
   another failure. Also reset openedAt to 0 in recordSuccess when the
   state recovers to healthy, so a stale timestamp cannot linger.

2. agent/loop/AgentLoop.ts: replace three silent '.catch(() => {})'
   sites (PreModelRequest, InstructionsLoaded, onCompactPersisted) with
   logged catches. Lifecycle hooks and the onCompactPersisted callback
   are user-configured extensions; silently swallowing their failures
   masks misconfigured hooks and lost writes. The remaining
   '.catch(() => {})' sites in this file handle shutdown / cleanup
   paths where logging would just be noise.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant