Skip to content

fix(responses): record first-output timing for tool-only streams (carry #6259) - #6392

Merged
lidge-jun merged 1 commit into
devfrom
codex/rel-l3-6259-tool-first-output
Oct 1, 2026
Merged

lidge-jun merged 1 commit into
devfrom
codex/rel-l3-6259-tool-first-output

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Summary

Carries #6259 by @xyjk0511 onto current dev so it can get full hosted CI. The single commit is replayed unchanged and keeps its author.

A Responses turn that produces only a tool call never recorded a first-output time, so the request log showed output tokens with an empty TTFT. First-output detection only counted text and reasoning deltas. This PR also counts non-empty tool input:

  • Native Responses SSE (firstOutputFromParsed in src/server/relay.ts) now counts non-empty response.function_call_arguments.delta and response.custom_tool_call_input.delta.
  • The adapter bridge (src/bridge/sse.ts) counts a tool_call_delta with non-empty arguments.

Empty deltas, tool-start scaffolding, lifecycle events, and steer/inject control frames still do not start the timer, and the timestamp is still reported once. The dashboard guide and the transport docs now say the value is the first output the proxy saw, not the start of hidden reasoning or a decode-speed measurement. Whitespace-only tool input counts as output, the same length > 0 rule that text and reasoning already use.

Carried from #6259. Original author: @xyjk0511.

Co-authored-by: xyjk buchanliang@gmail.com

Verification

  • I did not run local test suites or typecheck, on the project owner's explicit instruction. This is separate from the AGENTS.md resource exception. Hosted CI at the exact head of this PR is the only execution evidence. The original fork PR only ran the intake checks, so the full suite has not run on this change before.
  • The carry is git cherry-pick f2771cfe5c..6f44cd646e onto dev 7b2deb8059. It applied without conflicts: 7 files, +64/-7, identical to fix: record first-output timing for tool-only Responses streams #6259.
  • Static checks run locally: git diff --check, bun run scripts/privacy-scan.ts, and bun run structure:check pass. None of the touched files is in tests/fixtures/file-size-baseline.json. tool_call_delta.arguments is typed string in src/types/request.ts, so the new branch reads a defined field.
  • New regression cases are in tests/adapters/bridge.test.ts and tests/server/response-log-inspection.test.ts, both existing files.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. This change touches no auth, credential, or request-body logging path; it only changes when a timing hook fires.

Summary by CodeRabbit

  • Improvements

    • First-output timing now includes non-empty function arguments and custom-tool input, so tool-only responses can register a first output.
    • Empty input fragments and tool-start notifications do not start the timer. Timing reflects the first output observed by the proxy, not hidden reasoning or exact model-decoding speed.
  • Documentation

    • Updated dashboard and response-transport guides to explain first-output timing behavior.

Constraint: Keep the existing once-only callback and empty/control-event exclusions in native and adapted Responses streams.
Rejected: Match every event ending in .delta | control and echo payloads must not start output timing.
Confidence: high
Scope-risk: narrow
Directive: First-output timing is a proxy observation, not the start of hidden reasoning or exact model decoding.
Tested: 183 focused Bun tests; TypeScript typecheck; privacy scan; structure checks; documentation build; pre-fix regressions reproduce missing timing.
Not-tested: Full repository suite; upstream live inference on this dev checkout.
@lidge-jun
lidge-jun requested a review from Ingwannu as a code owner October 1, 2026 12:54
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-01T12:57:43.591978Z 3937885 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used
📚 Code guidelines (2)
src/AGENTS.md — auto-discovered
structure/AGENTS.md — auto-discovered

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: lidge-jun/opencodex/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: d82c06aa-02a6-455e-ba8a-e36daaafc165

📥 Commits

Reviewing files that changed from the base of the PR and between 7b2deb8 and 3937885.

📒 Files selected for processing (7)
  • docs-site/src/content/docs/guides/web-dashboard.md
  • src/bridge/sse.ts
  • src/server/relay.ts
  • structure/transports/responses-wire-shapes.md
  • structure/transports/responses.md
  • tests/adapters/bridge.test.ts
  • tests/server/response-log-inspection.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.


📝 Walkthrough

Walkthrough

First-output timing now starts on nonempty function-argument and custom-tool-input deltas. Empty deltas and tool-start scaffolding do not start the timer. Tests and documentation cover these event rules.

Changes

First-output timing

Layer / File(s) Summary
Recognize tool-input deltas
src/bridge/sse.ts, src/server/relay.ts, tests/adapters/bridge.test.ts, tests/server/response-log-inspection.test.ts, structure/transports/responses.md, structure/transports/responses-wire-shapes.md, docs-site/src/content/docs/guides/web-dashboard.md
The bridge and relay treat nonempty function-argument and custom-tool-input deltas as first output. Tests check empty and non-string deltas, fragmented input, and one-shot callback behavior. Documentation describes which events start the timer and what the measurement represents.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 39378

No actionable merge-blocking risk is identified. The change adds first-output timing for nonempty tool-input deltas, with tests and documentation describing the behavior.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 4 files. (3 skipped: 3… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: recording first-output timing for Responses streams that contain only tool calls. The carry reference is additional context and does not redu…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 4 files. (3 skipped: 3 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lidge-jun

Copy link
Copy Markdown
Owner Author

Maintainer integration record (MAINTAINERS.md, dev-only) — DRAFT from lane L3

@lidge-jun
lidge-jun merged commit a207452 into dev Oct 1, 2026
35 checks passed
@lidge-jun
lidge-jun deleted the codex/rel-l3-6259-tool-first-output branch October 1, 2026 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant