Skip to content

[Bug]: a spent Codex plan window is cooled for 60s because it arrives as HTTP 502, not 429 — 942 doomed sends in 72h #5860

Description

@vadymhimself

Client or integration

Claude Code

Area

Proxy and routing

Summary

A depleted ChatGPT Codex plan window is cooled for 60 seconds instead of a window-length hold, so an exhausted account is re-sent to roughly once a minute for the life of the window. Over 72 hours of one production combo this produced 942 doomed sends — 847 openai, 95 combo — each surfacing to the caller as adapter_eof, plus 94 downstream 503 No available targets for combo.

The cause is a classification miss, not the cooldown policy itself. The Codex backend reports a spent window as HTTP 502 upstream_server_error carrying the prose The usage limit has been reached — never the documented HTTP 429. isProviderScopedQuotaCap (src/combos/failover.ts) requires status === 429 && (normalizedCode === "gousagelimiterror" || text.includes("monthly usage limit reached")), so an exhausted window never matches it. The target falls through coolComboTarget's duration chain to DEFAULT_COOLDOWN_MS and is offered again 60 seconds later, where it fails identically.

This is invisible in normal operation: the combo hops to a healthy target and the request still returns 200. The cost is wasted upstream calls and latency, not user-visible errors.

The design question, which is why this is an issue and not a PR

The obvious fix is a status-independent match on the exhaustion prose, used only to choose the cooldown duration. That collides with an existing expectation:

tests/codex-integration/combos.test.ts → keeps the default cooldown for usage-window 1308 asserts that code 1308 with message Usage limit reached for 5 hour is still cooling at +59,999 ms and no longer cooling at +60,000 ms. It arrived in 6b2dfde11 (#3294) as the negative control proving 1308 does not get the new 5-second request-rate cooldown, so the 60-second value reads as a side effect of that contrast rather than a deliberate position on exhaustion.

Since 1308 is itself a 5-hour usage window, the same reasoning applies to it: a 60-second cooldown re-probes a 5-hour window about 300 times before it can possibly succeed.

So the question for maintainers: is 60 seconds intended for a spent usage window, or is that pin an artifact of the 1308-vs-request-rate contrast?

  • If it is an artifact, I will send a PR making a known-exhausted window take MAX_COOLDOWN_MS (10 minutes, already the clamp on this path) and updating that test, with the reasoning recorded.
  • If 60 seconds is intended, I will drop it and adapt downstream instead.

Ten minutes rather than a pin to the advertised reset is deliberate: quota commonly frees earlier than advertised (#433), and a longer hold would skip a target that had already recovered.

Scope would be duration only. isProviderScopedQuotaCap, comboFailureCooldownScope, comboFailureDecision and isTransientRequestRateLimit would be untouched, so a Codex 502 still resolves to target scope and hop through the existing status >= 500 path. A status-blind prose match is safe for choosing how long to wait and would not be safe for choosing what to black out, which is why it would be read in exactly one place.

A sibling case — permanent credential and billing failures in PROVIDER_SCOPED_FAILURE_CODES taking the same 60-second default — has no such collision and is filed separately as #5859.

Reproduction

  1. Configure a failover combo with a ChatGPT Codex target and one healthy target.
  2. Drive the Codex account's plan window to exhaustion.
  3. Send requests continuously through the combo.
  4. Observe in ocx logs --json: attempts against the exhausted target with sendCount > 0 recurring at roughly 60-second intervals for the life of the window, each failing, while the request itself succeeds on the sibling target.

Version

2.59.1

Operating system

macOS 15 (Darwin 25.6.0), Apple Silicon

Provider and model

openai / gpt-6-astra (ChatGPT Codex backend), in a two-target failover combo alongside anthropic / claude-opus-5

Logs or error output

Aggregated from the request ledger over 72 hours; no prompt content or credentials included.

doomed sends (attempt sendCount>0, rejected, target already known exhausted): 942
  by provider: openai 847, combo 95
caller-visible symptom on those attempts: adapter_eof
downstream: 94 x 503 "No available targets for combo: waterfall"
wire classification on the exhausted target: HTTP 502, code upstream_server_error,
  message contains "The usage limit has been reached"

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingproxyHTTP proxy, routing, reverse-proxy / management auth

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions