Skip to content

Fix: Centralize Codex priority pricing#2175

Open
iam-brain wants to merge 1 commit into
steipete:mainfrom
iam-brain:iam-brain/codex-priority-multipliers
Open

Fix: Centralize Codex priority pricing#2175
iam-brain wants to merge 1 commit into
steipete:mainfrom
iam-brain:iam-brain/codex-priority-multipliers

Conversation

@iam-brain

@iam-brain iam-brain commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • centralize Codex Priority multipliers in one model-aware helper
  • apply 2x to GPT-5.4 and 2.5x to GPT-5.5 and newer models, including GPT-5.6 Sol, Terra, and Luna
  • resolve Standard prices from models.dev first, with bundled legacy pricing as fallback
  • include the multiplier contract in scanner cache fingerprints so cached totals are invalidated when pricing changes

Multiplier contract and sources

Codex model generation Priority multiplier Source
GPT-5.4 2x Standard OpenAI Codex Speed documentation
GPT-5.5 2.5x Standard OpenAI Codex Speed documentation and GPT-5.5 launch pricing
GPT-5.6 2.5x Standard OpenAI Codex Speed documentation

The Codex Speed documentation directly states: GPT-5.6 and GPT-5.5 consume at 2.5x the Standard rate, while GPT-5.4 consumes at 2x.

The GPT-5.6 value was initially deduced from its continuation of GPT-5.5 pricing: GPT-5.5 is $5 input / $30 output per million tokens, and GPT-5.6 Sol is also $5 input / $30 output per million tokens. The current Codex Speed documentation now directly confirms that deduction by explicitly assigning GPT-5.6 the same 2.5x multiplier as GPT-5.5.

Behavior proof

models.dev-first calculation

The focused pricing fixture supplies a models.dev Standard total of $0.000815 for GPT-5.6 Sol. The helper returns:

Standard (models.dev): $0.000815
Priority multiplier:   2.5x
Priority result:        $0.0020375

This proves the multiplier is applied after models.dev resolution rather than to stale bundled rates.

End-to-end scanner calculation

A redacted deterministic Codex Priority trace with 100 input tokens (20 cached) and 10 output tokens is scanned through CostUsageScanner.loadDailyReport. With the injected models.dev GPT-5.5 rates:

Scanner Standard total: $0.000890
Priority multiplier:    2.5x
Scanner Priority total: $0.002225
Reported model split:   priorityCostUSD = $0.002225

The scanner test exercises the session JSONL + Priority trace metadata join, models.dev price resolution, model breakdown, daily aggregation, and cache-writing path. Separate tests cover bundled legacy fallback, cached Priority surcharge reuse, cache invalidation, GPT-5.6 cache-write pricing, and the 272K long-context guard.

Validation

  • swift test --filter CostUsagePricingTests - 46 passed
  • swift test --filter CostUsageScannerPriorityTests - 17 passed
  • make check - passed
  • SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.5.sdk make test - 638 selections passed across 54 groups
  • upstream CI - all macOS, Linux, lint, build, and security checks passed
  • review gate - no material findings

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Jul 15, 2026
@clawsweeper

clawsweeper Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed July 16, 2026, 7:56 PM ET / 23:56 UTC.

Summary
The PR replaces explicit per-model Codex Priority rates with generation-based multipliers over models.dev-first Standard pricing, updates scanner cache fingerprinting, and adds focused pricing and scanner tests.

Reproducibility: yes. at the source-contract level: the PR multiplies Standard USD prices while the official documentation distinguishes ChatGPT credit consumption from API token billing. A high-confidence real-session reproduction is still missing.

Review metrics: 3 noteworthy metrics.

  • Pricing surface: 5 files changed. The patch spans the pricing model, scanner cache reuse and writes, generated parser fingerprint, and two focused test suites.
  • Diff size: 133 additions, 76 deletions. This is a bounded refactor, but it replaces the source of truth for every affected Priority cost calculation.
  • Validation checks: 8 check runs passed. Automated checks support implementation quality but do not settle the accounting contract or substitute for real-session proof.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🦐 gold shrimp
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add redacted real Fast-session logs or terminal output showing classification, calculated total, and displayed result; remove private endpoints, account details, keys, and other sensitive information.
  • Resolve the ChatGPT-credit versus API-token accounting decision in the PR description and implementation.
  • Rebase or merge current main to resolve the dirty merge state.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR provides deterministic fixtures and scanner-test output only; add redacted terminal output or logs from a real after-fix Fast session, update the PR body to trigger review, and ask a maintainer for @clawsweeper re-review if it does not retrigger automatically.

Risk before merge

  • [P1] The displayed USD totals may become systematically misleading if ChatGPT credit-consumption multipliers are treated as token-price multipliers without an explicit product contract.
  • [P1] The supplied proof does not demonstrate how a real signed-in Fast session is classified, priced, cached, and displayed after the change.
  • [P1] The current head has merge conflicts and requires refresh against current main before it can land.

Maintainer options:

  1. Clarify and split accounting before merge (recommended)
    Document and implement distinct credit-estimate and API token-pricing semantics, then add redacted real-session evidence for the selected path.
  2. Accept a heuristic USD estimate
    Merge only after the owner explicitly accepts that Standard-rate multiplication is an estimate rather than an official USD Fast-pricing contract.
  3. Preserve current explicit pricing
    Pause or close this branch if the accounting distinction cannot be made stable and user-comprehensible.

Next step before merge

  • [P1] The owner should first decide what the USD metric represents; after that, the contributor must add real-session proof and resolve the current merge conflicts.

Maintainer decision needed

  • Question: Should CodexBar translate ChatGPT Fast credit multipliers into estimated USD by multiplying models.dev Standard rates, or retain explicit per-model USD rates until an official dollar-pricing contract exists?
  • Rationale: Official documentation now establishes credit-consumption multipliers but distinguishes that mechanism from API-key token billing, so source inspection alone cannot determine what CodexBar intends its USD metric to represent.
  • Likely owner: steipete — The repository owner already stated the acceptance boundary and requested the authoritative pricing basis in the PR discussion.
  • Options:
    • Separate credit and token accounting (recommended): Model signed-in Fast usage as a documented credit-derived estimate while retaining API-specific token pricing wherever the session type can be identified.
    • Accept multiplier-derived USD estimates: Explicitly define the displayed Priority USD amount as a heuristic derived from Standard rates and require UI/docs wording plus real-session proof.
    • Retain explicit rates: Keep the current per-model Priority pricing and close or narrow this PR until a stable official USD mapping is available.

Security
Cleared: The five-file patch adds no dependency, permission, secret-handling, download, publishing, or third-party execution changes.

Review details

Best possible solution:

Define separate, documented accounting semantics for signed-in ChatGPT Fast credit estimates and API-key Priority token costs, preserve existing explicit rates until that boundary is approved, and validate the chosen path with a redacted real-session trace plus the displayed CodexBar result.

Do we have a high-confidence way to reproduce the issue?

Yes at the source-contract level: the PR multiplies Standard USD prices while the official documentation distinguishes ChatGPT credit consumption from API token billing. A high-confidence real-session reproduction is still missing.

Is this the best way to solve the issue?

Unclear. Centralizing the calculation and invalidating caches are maintainable choices, but one multiplier helper is not the best solution unless maintainers confirm that credit-derived and API-priced usage should share the same displayed USD semantics.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a1fad485d668.

Label changes

Label justifications:

  • P2: Incorrect cost estimates affect a bounded reporting feature with meaningful user impact but do not threaten app availability or user data.
  • merge-risk: 🚨 other: Merging could change all affected Priority USD totals based on an unresolved distinction between ChatGPT credit consumption and API token billing.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR provides deterministic fixtures and scanner-test output only; add redacted terminal output or logs from a real after-fix Fast session, update the PR body to trigger review, and ask a maintainer for @clawsweeper re-review if it does not retrigger automatically.
Evidence reviewed

What I checked:

  • Official contract distinguishes accounting modes: The Codex Speed documentation states that GPT-5.5 and GPT-5.6 consume ChatGPT credits at 2.5× Standard and GPT-5.4 at 2×, while API-key usage is billed through API token pricing with separate Priority processing rates. citeturn1view0.
  • Current product surface: Current-main documentation describes cost usage as a local estimate built from Codex session logs and Priority traces, so the meaning of the displayed USD amount must remain explicit when translating subscription-credit behavior. citeturn2view1. (docs/codex.md, a1fad485d668)
  • Patch accounting behavior: The proposed helper removes explicit Priority token rates and calculates Priority cost by multiplying resolved Standard USD pricing, including models.dev values, according to model generation. (Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift, 763bced20698)
  • Maintainer direction remains unresolved: The repository owner asked for a stable authoritative contract mapping the rates to models and explicitly requested retaining current per-model rates until that basis is established; the later official source establishes credit multipliers but leaves the credit-to-USD interpretation unresolved.
  • Only deterministic proof supplied: The PR body reports fixture and scanner-test calculations, but no redacted output or logs from an actual signed-in Codex Fast session demonstrate the after-fix displayed result. (Tests/CodexBarTests/CostUsageScannerPriorityTests.swift:53, 763bced20698)
  • Relevant feature history: CodexBar release history attributes the original Fast/Priority pricing and model-breakdown work to prior merged contributions from iam-brain, while later GPT-5.6 pricing work is attributed to 0xSMW. citeturn2view2turn2view3. (CHANGELOG.md, a1fad485d668)

Likely related people:

  • steipete: The owner explicitly set the acceptance boundary in the discussion and is the best available decision owner for whether credit multipliers may define the displayed USD estimate. (role: repository owner and current reviewer; confidence: high; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift)
  • iam-brain: Beyond proposing this branch, prior merged release history credits this contributor with the original Fast/Priority pricing, cost-history, and model-breakdown behavior. citeturn2view2turn2view3. (role: feature owner and recent area contributor; confidence: high; commits: 763bced20698; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift, Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner+CacheHelpers.swift, Tests/CodexBarTests/CostUsagePricingTests.swift)
  • 0xSMW: Release history credits this contributor with the current GPT-5.6 model-family pricing work that this PR would replace with multiplier-based derivation. citeturn2view2. (role: recent pricing contributor; confidence: medium; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (4 earlier review cycles)
  • reviewed 2026-07-15T08:57:45.391Z sha 763bced :: needs real behavior proof before merge. :: [P1] Use the published 2× multiplier for GPT-5.6
  • reviewed 2026-07-15T10:11:27.479Z sha 763bced :: needs real behavior proof before merge. :: [P1] Use the published 2× multiplier for GPT-5.6
  • reviewed 2026-07-16T02:07:29.577Z sha 763bced :: needs real behavior proof before merge. :: none
  • reviewed 2026-07-16T03:48:39.356Z sha 763bced :: needs real behavior proof before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jul 15, 2026
@iam-brain

Copy link
Copy Markdown
Contributor Author

@clawsweeper 2x is incorrect, as far as I know it's 2.5x for GPT-5.5 and newer models. Do you have a source for the 2x cost aside from the error in code?

@iam-brain

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

The PR description now includes the explicit Codex Fast-mode contract (2x for GPT-5.4; 2.5x for GPT-5.5 and newer), authoritative Codex-specific sources, models.dev-first arithmetic, and end-to-end scanner proof. The prior 2x finding used API Priority Processing pricing; this change accounts for Codex Fast mode.

@clawsweeper

clawsweeper Bot commented Jul 15, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@iam-brain

Copy link
Copy Markdown
Contributor Author

Correction to my prior comment: the API-rate-versus-credit-accounting distinction I asserted was unsupported and has been removed from the PR description. The intended contract is simply 2x for GPT-5.4 and 2.5x for GPT-5.5 and newer. Please evaluate the implementation and proof against that stated requirement.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. and removed merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Jul 16, 2026
@steipete

Copy link
Copy Markdown
Owner

Could you provide an authoritative source for these priority-pricing multipliers—ideally an official pricing page, documentation, or a stable product contract that maps the rates to each model?

Until that source is available, CodexBar should keep the existing explicit per-model rates. The previously asserted API-rate versus credit-accounting distinction was retracted, so it cannot support changing the pricing model on its own. Keeping this PR open for sourcing.

@iam-brain

iam-brain commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

5.5 was already inherently incorrect; OAI had that info available from launch to be 2.5x cost in priority mode. Latest docs reflect 5.6 is 2.5x as well, and was deducted using common sense beforehand since Sol follows 5.5's pricing.

PR description updated with multiple sources.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants