Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs-site/src/content/docs/reference/adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,11 @@ transport; it does not infer subscription attribution from the inbound protocol.
collects `usage`. Providers listed in `reasoningDetailsModels` (MiniMax M-series) instead read
structured `delta.reasoning_details` segments, whose `text` arrives as cumulative snapshots and
is prefix-diffed, and replay preserved reasoning as a `reasoning_details` array.
- Suppresses bare `<tool_call>` text when it duplicates a structured call. Two adjacent identical
blocks are also collapsed when one matching call has either a single input body or two copies
joined directly or by one newline. Repairing doubled input requires an arguments object with
only an `input` key. Trailing whitespace after the pair is suppressed; mismatched or example
markup remains visible.
- ClinePass uses the live-verified gateway format `reasoning: { enabled: true, effort }` (or
`{ enabled: false }` when reasoning is disabled); its public API docs do not currently specify
this request shape. The adapter preserves requested `low`, `medium`, `high`, `xhigh`, and `max`
Expand Down
32 changes: 32 additions & 0 deletions src/adapters/openai-chat/serialized-tool-call-content.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,15 @@ function callsIn(text: string, context: TextContext = { fence: null, lineStart:
return calls;
}

/** Recognizes one eligible block immediately echoed a second time, with no intervening text. */
function repeatedCallIn(text: string, context?: TextContext): SerializedToolCall | undefined {
const first = callsIn(text, context)[0];
if (!first) return undefined;
const block = text.slice(first.start, first.end);
if (text.slice(first.end).trimEnd() !== block.trimEnd()) return undefined;
return { ...first, end: text.length };
}

/** Splits safe visible text from a possible control block while carrying Markdown context across chunks. */
export function splitAtPossibleSerializedToolCall(
text: string,
Expand Down Expand Up @@ -286,6 +295,14 @@ function duplicatedSerializedToolCallRanges(
context?: TextContext,
): { start: number; end: number }[] {
if (structuredCalls.length === 0) return [];
const repeated = repeatedCallIn(text, context);
if (repeated) {
const matching = structuredCalls.filter(structured =>
structured.names.has(repeated.name)
&& inputFromArguments(structured.argumentsText)?.trimEnd() === repeated.body.trimEnd());
if (matching.length === 1) return [{ start: repeated.start, end: repeated.end }];
return [];
}
return callsIn(text, context).filter(call => {
const body = call.body.trimEnd();
return structuredCalls.some(structured =>
Expand Down Expand Up @@ -314,6 +331,21 @@ export function repairArgumentsDuplicatedBesideSerializedCall(
functionNames: ReadonlySet<string>,
serializedText: string,
): string {
const repeated = repeatedCallIn(serializedText);
if (repeated && functionNames.has(repeated.name)) {
const body = repeated.body.trimEnd();
try {
const parsed = JSON.parse(argumentsText) as unknown;
if (parsed !== null && typeof parsed === "object" && !Array.isArray(parsed)
&& Object.keys(parsed).length === 1
&& ((parsed as Record<string, unknown>).input === body + body
|| (parsed as Record<string, unknown>).input === body + "\n" + body)) {
return JSON.stringify({ input: body });
}
} catch {
// The existing malformed-JSON repair below may still apply.
}
}
try {
JSON.parse(argumentsText);
return argumentsText;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@
- Alternatives considered: Drop all tool-call-looking content, add a provider-specific switch, or reconcile serialized blocks with structured calls at the Chat adapter boundary.
- Choice: Hold only a possible complete markup block and suppress or repair it only when the function name and duplicated body agree with a structured call in the same response.
- Why: Agreement between both representations is deterministic and avoids changing ordinary commentary, mismatched markup, or unrelated providers' valid text.
- Consequences: Matching calls no longer appear twice; same-name/different-body examples remain visible; the small held region is translator-budgeted and emits heartbeats while held; terminal failures retain held text without dispatching tools; malformed concatenated arguments are repaired only for the exact duplicated wrapper shape.
- Consequences: Matching calls no longer appear twice; an exact pair of adjacent identical blocks with one doubled structured input is reduced to one call; same-name/different-body examples remain visible; the small held region is translator-budgeted and emits heartbeats while held; terminal failures retain held text without dispatching tools; malformed concatenated arguments are repaired only for proven duplicate shapes.
- Follow-up (260924): the streaming hold is bounded (8 KiB of prose after a closed block, 4 MiB total); past a bound held text is released unsuppressed. See structure/providers/chat-compat.md.
6 changes: 6 additions & 0 deletions structure/providers/chat-compat.md
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,12 @@ and removes the block only when its function name and
freeform body match a structured call's parsed `input` in the same response. If the gateway also prefixes the structured call's JSON
arguments with the same freeform body, the adapter keeps the JSON suffix only when the block body,
prefix, and wrapper's `input` value all agree. Mismatched markup and arguments remain byte-exact.
Two immediately adjacent identical bare blocks, with optional trailing whitespace, are suppressed
when exactly one structured call matches their function and has either one copy of their body as
`input`, or two copies joined directly or by one newline that can be reduced to one. Reducing a
doubled `input` requires an arguments object with no keys besides `input`; extra keys leave it
unchanged. Unrelated structured calls do not prevent suppression. Other repeated shapes remain
Comment thread
coderabbitai[bot] marked this conversation as resolved.
unchanged.
Silent held-content frames emit adapter heartbeats. Terminal errors and transport read failures
drain all held text, including matching serialized blocks, because pending tools are not dispatched.
The held bytes use the shared translator budget. The streaming hold is bounded (`ingestStreaming`): once a closed block is followed by more than 8 KiB of prose with no block open after it, or held text plus queued events would pass 4 MiB, everything held is released in order with nothing suppressed, so an unmatched block no longer delays the rest of the answer to the end of the turn. A duplicate is the tail of the content, so its reconciliation is unaffected; past either bound the stream prefers delivery (the pre-#5548 raw markup) over suppression. Buffered responses keep the unbounded `ingest` because their structured calls are already known (`tests/adapters/openai/openai-chat-serialized-tool-call-hold-bound.test.ts`). For a model opted into inline `<think>` splitting,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,159 @@ test("buffered Chat responses reconcile matching serialized and structured tool
});
});

test("buffered Chat responses reconcile two identical echoed blocks and doubled input", async () => {
const script = "const names = []; text(names);";
const block = `<tool_call><function=exec>${script}</parameter></function></tool_call>`;
const events = await createOpenAIChatAdapter(provider).parseResponse!(Response.json({
choices: [{
message: {
content: block + block,
tool_calls: [{
id: "call_exec",
function: { name: "exec", arguments: JSON.stringify({ input: script + script }) },
}],
},
finish_reason: "tool_calls",
}],
}), createTestTranslatorBudget());

expect(events.filter(event => event.type === "text_delta")).toEqual([]);
expect(events.find(event => event.type === "tool_call_delta")).toEqual({
type: "tool_call_delta",
arguments: JSON.stringify({ input: script }),
});
});

test("buffered Chat responses suppress two echoed blocks when structured input is already single", async () => {
const script = "text('ok');";
const block = `<tool_call><function=exec>${script}</parameter></function></tool_call>`;
const events = await createOpenAIChatAdapter(provider).parseResponse!(Response.json({
choices: [{
message: {
content: block + block,
tool_calls: [{
id: "call_exec",
function: { name: "exec", arguments: JSON.stringify({ input: script }) },
}],
},
finish_reason: "tool_calls",
}],
}), createTestTranslatorBudget());

expect(events.filter(event => event.type === "text_delta")).toEqual([]);
expect(events.find(event => event.type === "tool_call_delta")).toEqual({
type: "tool_call_delta",
arguments: JSON.stringify({ input: script }),
});
});

test("buffered Chat responses suppress two echoed blocks with a trailing newline", async () => {
const script = "text('ok');";
const block = `<tool_call><function=exec>${script}</parameter></function></tool_call>`;
const events = await createOpenAIChatAdapter(provider).parseResponse!(Response.json({
choices: [{
message: {
content: block + block + "\n",
tool_calls: [{ id: "call_exec", function: { name: "exec", arguments: JSON.stringify({ input: script }) } }],
},
finish_reason: "tool_calls",
}],
}), createTestTranslatorBudget());

expect(events.filter(event => event.type === "text_delta")).toEqual([]);
expect(events.find(event => event.type === "tool_call_delta")).toEqual({
type: "tool_call_delta", arguments: JSON.stringify({ input: script }),
});
});

test("buffered Chat responses repair two echoed blocks with newline-joined input", async () => {
const script = "text('ok');";
const block = `<tool_call><function=exec>${script}</parameter></function></tool_call>`;
const events = await createOpenAIChatAdapter(provider).parseResponse!(Response.json({
choices: [{
message: {
content: block + block,
tool_calls: [{ id: "call_exec", function: { name: "exec", arguments: JSON.stringify({ input: script + "\n" + script }) } }],
},
finish_reason: "tool_calls",
}],
}), createTestTranslatorBudget());

expect(events.filter(event => event.type === "text_delta")).toEqual([]);
expect(events.find(event => event.type === "tool_call_delta")).toEqual({
type: "tool_call_delta", arguments: JSON.stringify({ input: script }),
});
});

test("buffered Chat responses suppress a repeated echo beside an unrelated structured call", async () => {
const script = "text('ok');";
const block = `<tool_call><function=exec>${script}</parameter></function></tool_call>`;
const events = await createOpenAIChatAdapter(provider).parseResponse!(Response.json({
choices: [{
message: {
content: block + block,
tool_calls: [
{ id: "call_exec", function: { name: "exec", arguments: JSON.stringify({ input: script }) } },
{ id: "call_other", function: { name: "other", arguments: JSON.stringify({ input: "other" }) } },
],
},
finish_reason: "tool_calls",
}],
}), createTestTranslatorBudget());

expect(events.filter(event => event.type === "text_delta")).toEqual([]);
expect(events.filter(event => event.type === "tool_call_delta")).toEqual([
{ type: "tool_call_delta", arguments: JSON.stringify({ input: script }) },
{ type: "tool_call_delta", arguments: JSON.stringify({ input: "other" }) },
]);
});

test("buffered Chat responses preserve repeated markup when two structured calls match", async () => {
const script = "text('ok');";
const block = `<tool_call><function=exec>${script}</parameter></function></tool_call>`;
const content = block + block;
const argumentsText = JSON.stringify({ input: script });
const events = await createOpenAIChatAdapter(provider).parseResponse!(Response.json({
choices: [{
message: {
content,
tool_calls: [
{ id: "call_one", function: { name: "exec", arguments: argumentsText } },
{ id: "call_two", function: { name: "exec", arguments: argumentsText } },
],
},
finish_reason: "tool_calls",
}],
}), createTestTranslatorBudget());

expect(events.filter(event => event.type === "text_delta")).toEqual([{ type: "text_delta", text: content }]);
expect(events.filter(event => event.type === "tool_call_delta")).toEqual([
{ type: "tool_call_delta", arguments: argumentsText },
{ type: "tool_call_delta", arguments: argumentsText },
]);
});

test("buffered Chat responses preserve repeated markup when the structured input differs", async () => {
const script = "text('example');";
const content = `<tool_call><function=exec>${script}</function></tool_call>`.repeat(2);
const argumentsText = JSON.stringify({ input: script + "text('other');" });
const events = await createOpenAIChatAdapter(provider).parseResponse!(Response.json({
choices: [{
message: {
content,
tool_calls: [{ id: "call_exec", function: { name: "exec", arguments: argumentsText } }],
},
finish_reason: "tool_calls",
}],
}), createTestTranslatorBudget());

expect(events.find(event => event.type === "text_delta")).toEqual({ type: "text_delta", text: content });
expect(events.find(event => event.type === "tool_call_delta")).toEqual({
type: "tool_call_delta",
arguments: argumentsText,
});
});

test("buffered Chat responses preserve serialized markup for a different function", async () => {
const content = "<tool_call><function=other>literal example</function></tool_call>";
const events = await createOpenAIChatAdapter(provider).parseResponse!(Response.json({
Expand Down
22 changes: 18 additions & 4 deletions tests/responses/responses-chat-tool-call-content.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,16 @@ afterEach(() => {
releaseSpendHome = undefined;
});

test("/v1/responses suppresses OpenAI Chat tool-call markup duplicated by a structured call", async () => {
async function checkEchoedToolCall(
repeated: boolean,
trailingNewline = false,
newlineJoinedInput = false,
): Promise<void> {
const savedFetch = globalThis.fetch;
const script = "const result = await tools.exec_command({cmd: \"pwd\"});\ntext(result.output);";
const leaked = `<tool_call><function=exec>${script}\n</parameter></function></tool_call>`;
const commentary = "I'll run it now.\n";
const content = commentary + leaked;
const content = commentary + leaked + (repeated ? leaked : "") + (trailingNewline ? "\n" : "");
const split = commentary.length + 5;
const frames = [
{ choices: [{ delta: { content: content.slice(0, split) } }] },
Expand All @@ -26,7 +30,12 @@ test("/v1/responses suppresses OpenAI Chat tool-call markup duplicated by a stru
tool_calls: [{
index: 0,
id: "call_exec",
function: { name: "exec", arguments: script + JSON.stringify({ input: script }) },
function: {
name: "exec",
arguments: repeated
? JSON.stringify({ input: script + (newlineJoinedInput ? "\n" : "") + script })
: script + JSON.stringify({ input: script }),
},
}],
},
}],
Expand Down Expand Up @@ -85,4 +94,9 @@ test("/v1/responses suppresses OpenAI Chat tool-call markup duplicated by a stru
} finally {
globalThis.fetch = savedFetch;
}
});
}

test("/v1/responses suppresses one echoed block", () => checkEchoedToolCall(false));
test("/v1/responses suppresses two echoed blocks with doubled input", () => checkEchoedToolCall(true));
test("/v1/responses suppresses trailing newline and repairs newline-joined doubled input", () =>
checkEchoedToolCall(true, true, true));
Loading