Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs-site/src/content/docs/guides/web-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,11 @@ host and port over a LAN IP or an alias.

## Dashboard layout

Responses first-output timing includes streamed function arguments and custom-tool input, as well
as text and reasoning. A tool-only turn can therefore have a first-output time even without prose.
Empty deltas and tool-start notifications do not start this timer. It measures the proxy's first
observed output, not the start of hidden model reasoning or exact model decoding throughput.

Overview uses matching status cards and full-width settings rows. On wide screens, labels share
one column and model/effort controls share another. On narrower screens, controls move below their
labels in the same reading order. Long version labels are shortened visually; hover the version
Expand Down
6 changes: 4 additions & 2 deletions src/bridge/sse.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ export function bridgeToResponsesSSE(
* response.completed — codex-rs collect_compaction_output requires exactly one.
*/
compaction?: boolean;
/** One-shot: first non-empty text/thinking/raw-reasoning delta observed (WP4 TTFT). */
/** One-shot: first non-empty text/thinking/raw-reasoning/tool-input delta observed. */
onFirstOutput?: () => void;
onTerminal?: (status: ResponsesTerminalStatus) => void;
onCompletedResponse?: (response: Record<string, unknown>, providerState?: OcxProviderContinuationState) => void;
Expand Down Expand Up @@ -707,7 +707,9 @@ export function bridgeToResponsesSSE(
? event.thinking.length > 0
: event.type === "reasoning_raw_delta"
? event.text.length > 0
: false;
: event.type === "tool_call_delta"
? event.arguments.length > 0
: false;
if (!nonEmpty) return;
firstOutputReported = true;
try { options?.onFirstOutput?.(); } catch { /* metrics must not break the stream */ }
Expand Down
4 changes: 3 additions & 1 deletion src/server/relay.ts
Original file line number Diff line number Diff line change
Expand Up @@ -779,7 +779,9 @@ export function firstOutputFromParsed(parsed: unknown): boolean {
const event = parsed as { type?: unknown; delta?: unknown };
return (event.type === "response.output_text.delta"
|| event.type === "response.reasoning_summary_text.delta"
|| event.type === "response.reasoning_text.delta")
|| event.type === "response.reasoning_text.delta"
|| event.type === "response.function_call_arguments.delta"
|| event.type === "response.custom_tool_call_input.delta")
&& typeof event.delta === "string"
&& event.delta.length > 0;
}
Expand Down
2 changes: 1 addition & 1 deletion structure/transports/responses-wire-shapes.md
Original file line number Diff line number Diff line change
Expand Up @@ -503,7 +503,7 @@ No pool timer or shutdown registration exists before eligible traffic activates
Translated response request-log tracking and the heartbeat relay also reuse
`createSseInspector`. This keeps every client-facing SSE observation path on
the same byte-bounded, discard-and-resynchronize frame policy and ensures the
request-log, first-output, and terminal observers share one payload parse.
request-log, first-output, and terminal observers share one payload parse. First-output timing recognizes nonempty text, reasoning, function-argument, and custom-tool-input deltas; empty deltas, tool scaffolding, control/echo frames, and terminal snapshots do not start it.
The inspector records a structured `response.failed` status before invoking the
terminal observer. Native Responses, Chat Completions, Claude Messages, and WebSocket
request logs must therefore finalize through the context-aware terminal mapper; recognized
Expand Down
2 changes: 1 addition & 1 deletion structure/transports/responses.md
Original file line number Diff line number Diff line change
Expand Up @@ -576,7 +576,7 @@ reuses `beginInferenceAttempt` and `createFinalRequestLog` with its own 401/429
## Adapter-to-Responses bridge

`src/bridge.ts` is a re-export facade; the implementation lives in `src/bridge/`.
`src/bridge/sse.ts` (`bridgeToResponsesSSE`) turns adapter events into the Responses SSE stream,
`src/bridge/sse.ts` (`bridgeToResponsesSSE`) turns adapter events into the Responses SSE stream; its once-only first-output observer includes nonempty `tool_call_delta` arguments (including custom-tool input), but not tool-start scaffolding or empty arguments,
and `src/bridge/response-json.ts` (`buildResponseJSON`) builds the non-streaming Responses body
from the same events. `buildResponseJSON` records a buffered delivery on the attempt unless the
caller passes `recordBufferedDelivery: false`, which the direct client encoders do because they
Expand Down
23 changes: 21 additions & 2 deletions tests/adapters/bridge.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,17 +61,36 @@ describe("Responses bridge reasoning and usage parity", () => {
expect(firstOutputs).toBe(1);
});

test("first-output callback ignores tool-only streams", async () => {
test("first-output callback observes tool-only streams once", async () => {
let firstOutputs = 0;
await collectSse(bridgeToResponsesSSE(replay([
{ type: "tool_call_start", id: "call_1", name: "read_file" },
{ type: "tool_call_delta", arguments: "" },
{ type: "tool_call_delta", arguments: "{}" },
{ type: "tool_call_delta", arguments: " " },
{ type: "tool_call_end", id: "call_1" },
{ type: "done" },
]), "routed/model", undefined, undefined, undefined, undefined, undefined, {
onFirstOutput: () => { firstOutputs += 1; },
}));
expect(firstOutputs).toBe(0);
expect(firstOutputs).toBe(1);
});

test("first-output callback observes custom tool input but not empty tool scaffolding", async () => {
for (const input of ["", "synthetic input"]) {
let firstOutputs = 0;
const frames = await collectSse(bridgeToResponsesSSE(replay([
{ type: "tool_call_start", id: "custom_1", name: "probe_tool" },
{ type: "tool_call_delta", arguments: input },
{ type: "tool_call_end", id: "custom_1" },
{ type: "done" },
]), "routed/model", undefined, new Set(["probe_tool"]), undefined, undefined, undefined, {
onFirstOutput: () => { firstOutputs += 1; },
}));
expect(firstOutputs).toBe(input.length ? 1 : 0);
expect(frames.some(frame => frame.event === "response.completed")).toBe(true);
if (input) expect(frames.some(frame => frame.event === "response.custom_tool_call_input.delta")).toBe(true);
}
});

test("first-output callback still fires for hidden reasoning", async () => {
Expand Down
29 changes: 29 additions & 0 deletions tests/server/response-log-inspection.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,35 @@ import type { RequestLogContext } from "../../src/server/request-log";

const encoder = new TextEncoder();
const frame = (payload: unknown) => encoder.encode(`data: ${JSON.stringify(payload)}\n\n`);

describe("tool-output first timing", () => {
for (const type of ["response.function_call_arguments.delta", "response.custom_tool_call_input.delta"]) {
test(`${type} starts timing once through fragmented SSE`, () => {
let firstOutputs = 0;
const inspector = createSseInspector({ onFirstOutput: () => { firstOutputs++; } });
for (const event of [
{ type: "response.created" },
{ type: "response.output_item.added", item: { type: "function_call", arguments: "" } },
{ type: "response.steer.input.delta", delta: "echo" },
{ type: "response.inject.input.delta", delta: "echo" },
{ type, delta: "" },
{ type, delta: 42 },
]) inspector.feed(frame(event));
expect(firstOutputs).toBe(0);
for (const delta of ["{", " "]) {
const bytes = frame({ type, delta });
inspector.feed(bytes.subarray(0, 11));
inspector.feed(bytes.subarray(11));
}
expect(firstOutputs).toBe(1);
inspector.feed(frame({ type: "response.output_text.delta", delta: "later prose" }));
inspector.feed(frame({ type: "response.completed", response: { status: "completed", output: [] } }));
inspector.finish();
expect(firstOutputs).toBe(1);
expect(inspector.terminalSeen()).toBe(true);
});
}
});
const terminal = (id = "fixture-response") => ({
type: "response.completed",
response: {
Expand Down
Loading