feat(cookbooks): store threads in Gateway conversations and run tools with runTools()

[vishxrad]

This PR now includes #1261 (runTools() and raw completion chunks), since both changed the same chat routes. #1264 (chat library) is stacked on top.

Summary

All three cookbooks (conversational analytics, document comparison, and booking assistant) now keep their threads in Gateway conversations and run their tools with the OpenAI SDK's runTools().

Thread storage

• Browser storage. Agent Interface gets storage={useOpenuiCloudStorage({ token: "/api/frontend-token", features: { artifact: false } })}. The browser lists, creates, renames, and deletes threads directly with the Conversations API.
• Frontend token route. A new /api/frontend-token route mints a short-lived token with THESYS_API_KEY. It uses DEMO_USER_ID (default demo-user) and APP_ID (default per cookbook, for example booking-assistant-cookbook), the same variables apps from openui create use. The key stays on the server, and each cookbook sees only its own threads even when all three share a key.
• Turn storage. When each run ends, the chat route appends the turn to the thread's conversation with storeChatCompletionHistory() from @openuidev/server (new dependency, pinned to 0.1.0). The turn is the user's message, each tool call and its result, and the answer, collected from the runner's message events and stored once runner.done() settles. The conversation id is the threadId that Agent Interface's storage created. A stopped or failed run still stores the user's message and any tool steps that finished, so the question stays after a reload; a partly streamed answer isn't kept. A failed write is logged without affecting the answer. The response ends only after the write, so a follow-up finds the turn.
• History from the conversation. The route uses only the new question from the browser and loads the earlier turns, tool calls and results included, from the thread's Gateway conversation. src/lib/gateway-history.ts pages through GET /v1/conversations/{id}/items and converts the items back to Chat Completions messages, the reverse of storeChatCompletionHistory(), dropping tool calls that have no result (a stopped run). The browser can't supply this history: it never receives tool results over Chat Completions, so its copy has tool calls without results, which Gateway rejects with a 400. This way the model sees where each earlier answer came from, and conversation() in the route is the place to add compaction later.

Generation

• Tool loop removed. Each cookbook's src/lib/tool-loop.ts is gone. The chat route calls gateway.chat.completions.runTools() with the tool's JSON schema and its executor.
• The runner's own stream. The route returns runner.toReadableStream(), which carries every completion's chunks as one JSON object per line (application/x-ndjson), and the page reads it with openAIReadableStreamAdapter() and openAIMessageFormat. .asResponse() isn't available here: it belongs to a single create() call, while runTools() makes one request per round and reads each response itself to find the tool calls.
• Plain request handling. Like the other examples and the openui-cloud template, the routes read the body with request.json() and trust the local page. The chat route returns 400 without a threadId or a closing user message. The zod request schema, 1 MB body reader, origin checks, and 503s for a missing key or missing data are gone; a missing key or unprepared data now throws.
• Same round caps and first-round errors. maxChatCompletions is 3, 4, and 3. The route waits for the first chunk before responding, so a rejected key, rate limit, or unknown model returns an HTTP error with Gateway's message. Gateway reports an unknown model inside a successful response, so waiting only for the response let it through as a broken stream.

Docs

• Storage in one sentence. Each cookbook page covers thread storage in a one-sentence "Thread storage" callout that links storeChatCompletionHistory(), and each diagram shows the conversation. The pages don't describe the Conversations API or frontend tokens. The READMEs keep those details, including what to change before deploying: sign users in, mint tokens for the signed-in user, and check that a threadId belongs to that user before storing a turn in it.
• Walkthrough. The pages call generated output OpenUI Lang, name runTools() only in code and References, and drop paragraphs that repeated earlier steps. Each page ends with one References list of the packages, APIs, and guides the example uses. Mentions of authentication link to the Gateway authentication page, and the booking prompt no longer calls search_stays a Responses tool.
• Code samples fit the docs column. Sample lines are at most 80 characters and blocks at most 30 lines, so they don't scroll at desktop width. Document comparison's Agent Interface sample is split into its props and its slots. .prettierrc.json turns off embedded-code formatting for docs/content/docs/cookbooks/*.mdx, since Prettier would otherwise rewrap the samples at 100 characters. A few library.ts and sources.tsx sample lines here are still longer (up to 156 characters), because refactor(cookbooks): answer with the chat library #1264 replaces those samples.
• Diagrams. Stage labels are unnumbered, since the numbered steps under each diagram follow a different sequence.

What changes for readers

• No tool results in Behind the scenes during a run. Chat Completions has no chunk for a tool result, so Behind the scenes shows each call with its arguments but no result.
• Parallel tools. Tools the model calls in one completion run in parallel, and results go back to the model in call order.
• Tool errors. runTools() ends the run when a tool throws, so each route returns the error to the model as the tool result.
• Later errors are generic. An error after the first chunk ends the stream, and Agent Interface shows "Something went wrong: Failed to fetch" instead of Gateway's message.
• No forced final answer. A run that hits maxChatCompletions on a tool call ends without an answer.

Gateway items endpoint (fixed)

storeChatCompletionHistory() posts to POST /v1/conversations/{id}/items. Until 30 Sep 2026, Gateway answered valid requests with 404 "Conversation not found" for conversations that every other endpoint found. It now returns 200, the repro script prints "Not reproduced", and a reloaded thread loads its messages.

Test plan

• npm ci and npm run verify (spec generation and next build with type checking) in all three cookbooks at this PR's head, rebased on main
• In the browser, all three: the token route returns 200, GET /v1/conversations returns 200, and the first message creates a conversation (201) and streams the answer into a new thread
• With toReadableStream(), in the browser: analytics streams the pre-tool text, the query_lap_times call, and the lap-by-lap answer; document comparison shows two search_documents calls in one round and the page-cited answer; booking streams the prefilled trip form
• Invalid THESYS_API_KEY: /api/chat returns HTTP 401 with Gateway's message
• Unknown OPENUI_MODEL (booking): /api/chat returns HTTP 502 with {"error":"Unsupported model: …"} instead of a 200 stream that fails; a normal request still streams (200, 709 chunks)
• After simplifying the routes, in the browser: analytics answers the fastest-laps starter with a query_lap_times call, a table, and a chart, and after a reload the reopened thread loads its question, tool call, and answer; document comparison answers "Compare revenue and growth" with page-cited sources; booking returns the prefilled New York form. No server errors.
• With history loaded from Gateway, in the browser: analytics answers a question and a follow-up, and the converted history for that thread is user, assistant with tool_calls, the matching tool result, and the answer, for each turn; document comparison extends a revenue and R&D comparison with headcount; booking goes from the prefilled form to stay cards to the summary. No server errors.
• After simplifying: /api/chat returns 400 without a threadId or when the last message isn't the user's, and the token route returns 200 with Cache-Control: private, no-store
• A token from the route creates conversations with user_id: demo-user and the cookbook's app_id
• Probe: a token for another user_id or app_id gets 404 on the conversation and its items
• After Gateway fixed the items endpoint: a reloaded booking thread loads its question and prefilled form from GET …/items (200)
• Stopped runs, against real Gateway storage: cancelled before any answer stores the user's message; cancelled mid-answer (after 50 chunks) stores the user's message; stopped after a tool finished stores the message, tool call, and result; an unknown model stores the user's message
• Docs site, locally at refactor(cookbooks): answer with the chat library #1264's head on the rebased stack: the three cookbook pages render with no console or server errors, each "Thread storage" callout is one sentence, the diagrams have no stage numbers, and no code block is taller than its 600px box
• At this PR's head, no cookbook sample is over 30 lines, and Prettier passes on the three cookbook pages

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
openui-docs	Ready	Preview	Sep 30, 2026 2:02pm UTC