Skip to content

fix(codex): refresh validated snapshots during catch-up - #4087

Open
steipete wants to merge 1 commit into
mainfrom
triage/20260921-codex-day-attribution
Open

steipete wants to merge 1 commit into
mainfrom
triage/20260921-codex-day-attribution

Conversation

@steipete

Copy link
Copy Markdown
Owner

Codex catch-up kept a one-time publication flag after showing its first validated window. Later bounded passes could update that window while historical work remained pending, but the menu and widget retained the earlier snapshot. Retry the existing guarded cache publication after each pending pass.

The regression checks that a validated $4 result replaces $2 before the next automatic sleep, then allows the final authoritative correction to $1. Added synthetic coverage for event-time bucketing over three local days, active/archive copies, parser-revision migration, and discovery completion. Production line count is unchanged.

Validation used a temporary SwiftPM harness containing the repository's app/core sources and focused test files, with byte-identical Package.resolved pins:

CODEXBAR_SUPPRESS_TEST_KEYCHAIN_ACCESS=1 CODEXBAR_TEST_CODEX_FILE_ISOLATION=1 CODEXBAR_TEST_SESSION_FILE_ISOLATION=1 \
swift test --package-path .build/focused-cost-tests --scratch-path .build --force-resolved-versions --jobs 2 \
  --filter 'UsageStoreCodexCostCatchUp|CodexDayAttributionTests'

Before the fix: 16 passed, one failed because the worker still showed $2 instead of $4. After the fix: 28 passed, one opt-in benchmark skipped, zero failures across three suites. The opt-in 1,500-file/150,000-event benchmark also passed. The isolated P2 review found no actionable issues.

Refs #3508
Refs #3303
Refs #3420

TMPDIR="$PWD/.build/task-tmp" make check passed: Found 0 violations, 0 serious in 2671 files. Four pre-existing SQLite lock tests that timed out during shared filesystem pressure passed on isolated retry (4/4). The final broad check also passed its process-cleanup and cache-writer tests.

Publish completed-window cache results after each pending bounded pass,
so an earlier publication cannot suppress later updates. Keep existing
scope and completeness checks and final downward reconciliation.

Cover resumed sessions across local days, active/archive copies, parser
migration, and discovery completion with synthetic fixtures.

Refs #3508, #3303, #3420.
@clawsweeper

clawsweeper Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 28, 2026
@clawsweeper

clawsweeper Bot commented Sep 28, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed September 27, 2026, 11:23 PM ET / September 28, 2026, 03:23 UTC.

ClawSweeper review

What this changes

The branch makes CodexBar refresh validated Codex cost snapshots after each pending catch-up pass and adds synthetic regression coverage and documentation for that behavior.

Merge readiness

✅ Ready for maintainer review

Current main and v0.68.0 can retain an earlier Codex cost snapshot after a later scan pass validates newer totals. This PR addresses that remaining publication gap. The broader catch-up investigation and the separate counter-reset reports remain open.

Priority: P2
Reviewed head: bf657446099f03ae5a5b3ec543551f69b479395f

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A narrow, guarded fix has focused regression coverage and no concrete review finding; live app behavior was not independently observed.
Proof confidence 🌊 off-meta tidepool Not applicable: The changed UsageStore publication path is covered by a reported synthetic before/after test showing $2, then $4 before sleep, then a final $1; no packaged-app observation is supplied. The PR is owner-authored, so the external-contributor proof gate does not apply. No stored-data contract changes.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: The changed UsageStore publication path is covered by a reported synthetic before/after test showing $2, then $4 before sleep, then a final $1; no packaged-app observation is supplied. The PR is owner-authored, so the external-contributor proof gate does not apply. No stored-data contract changes.
Evidence reviewed 9 items Introduced production change: The pinned PR delta removes the one-time guard from publication after a pending bounded pass; it leaves the guarded snapshot loader and final reconciliation in place.
Current-main gap: Fetched main still skips publication after later pending passes once the first validated window has been published. The same guard is present in the v0.68.0 source.
Publication safeguards: The existing publisher checks cancellation, current scope and publication revision, established history, and stale-cache state before updating the Codex snapshot and persisting the widget.
Findings None None.
Security None None.

How this fits together

CodexBar scans local Codex session files into a cost cache. Its usage store publishes validated cache snapshots to the menu and widget while bounded historical scanning continues.

flowchart LR
  A[Codex session files] --> B[Bounded cost scan]
  B --> C[Local cost cache]
  C --> D{Reporting window validated?}
  D -->|Yes| E[Usage store snapshot]
  D -->|No| F[Retain prior snapshot]
  E --> G[Menu and widget]
  F --> G
Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test lines production +0 net (1 added, 1 removed); tests +156 net The behavior change is one guard edit with focused synthetic coverage.

Technical review

Best possible solution:

Publish each newly validated same-scope reporting window during catch-up, retain incomplete history, and let the final complete snapshot reconcile totals in either direction.

Do we have a high-confidence way to reproduce the issue?

Yes. The current-main guard and the focused before/after assertion establish a deterministic source-level path, though this read-only review did not run the test.

Is this the best way to solve the issue?

Yes. Reusing the existing completed-cache and scope checks is a narrow repair that preserves final downward reconciliation.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 579f68406855.

Labels

Label changes:

  • add P2: Stale Codex cost totals affect usage monitoring, while quota refresh and the rest of the app remain available.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The changed UsageStore publication path is covered by a reported synthetic before/after test showing $2, then $4 before sleep, then a final $1; no packaged-app observation is supplied. The PR is owner-authored, so the external-contributor proof gate does not apply. No stored-data contract changes.

Label justifications:

  • P2: Stale Codex cost totals affect usage monitoring, while quota refresh and the rest of the app remain available.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The changed UsageStore publication path is covered by a reported synthetic before/after test showing $2, then $4 before sleep, then a final $1; no packaged-app observation is supplied. The PR is owner-authored, so the external-contributor proof gate does not apply. No stored-data contract changes.

Evidence

What I checked:

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • kernnel: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant