Count Codex usage in the ledger, billed per turn - #89
Draft
ndr-ds wants to merge 4 commits into
Draft
Conversation
…ts' into ndr-ds/codex-usage-ledger
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The ledger only ever read
~/.claude/projects. Codex sessions contributed nothing to today/week/month, and nothing to session cost —has_usage_metrics()is false for them, so #79 correctly renders the total as a floor. Honest, but it means half the fleet is invisible.What changed
Codex rollouts under
$CODEX_HOME/sessions(default~/.codex/sessions) are now scanned into the same ledger, so the windows cover both products.Codex reports a running total, not per-message usage.
total_token_usageis cumulative for the session, and a row pertoken_countevent would re-bill everything spent so far on every turn — the same over-counting that made the Claude ledger read 11x high. So turns are the differences between successive snapshots, which sum exactly to the session total.last_token_usagelooks like it would do the job, but it is only present on some events and nothing guarantees the series partitions the total. Differencing the cumulative figure needs no such assumption.Details worth knowing:
codex:<session>:<ordinal>. The rollout line'sordinalis stable across re-reads, so re-scanning is idempotent — the same property that makes the Claude side safe.turn_contextis re-emitted per turn and a session can switch models mid-flight; attributing the whole session to whatever it ended on would misprice it.Verified
New tests: cumulative totals become per-turn deltas and sum back to the session total; per-turn model attribution across a mid-session switch (
gpt-5-codex→gpt-5.4); ordinals stable across reads; a truncated (still-being-written) rollout yields its complete turns; rollouts reach the ledger; a re-scan changes no total; Claude and Codex usage share one ledger (1 Claude message + 3 Codex turns = 4, tokens summed across both); and Codex turns priced from the feed rather than the fallback.A
rollout-multi-turn.jsonlfixture is checked in beside the existing one — the shipped fixture has a singletoken_countevent and cannot exercise differencing at all.cargo clippy --all-targetsandcargo fmt --checkclean; full suite green on 3 consecutive runs.Notes for review
gpt-5-codexhas no static entry and falls back to the flagship-Opus tier, ~4x its real $1.25/$10. Both are merged into this branch, so the diff to review here is the last commit.input_tokens/output_tokens, so cached input bills at the full input rate. LiteLLM does carrycache_read_input_token_costforgpt-5-codex, so if the rollout ever grows a cached-token field this gets more accurate for free. Today it slightly over-reports a heavily-cached Codex session.find_jsonl_files_cachedended withcache.retain(|k, _| seen_subdirs.contains(k)), butseen_subdirsonly describes the root just scanned while the cache is global — so scanning a second root evicted the first root's entries. Harmless in production (one root) and invisible until a test scanned two, which is how it surfaced. Now scoped to children ofroot.drained. A live session's rollout keeps growing and there is no writer-liveness signal here, so they are gated on mtime+size instead. That is one stat per rollout per scan — cheap at Codex's file counts, but it is a real difference from the Claude path if the rollout count ever grows to Claude's scale.