fix(openai-agents): accept sync tool handlers - #119
Merged
Merged
Conversation
The OpenAI agent handler awaited every tool handler result. A sync tool handler returns a plain value, so the call raised a TypeError and the agent run failed. Call the tool handler, then await the result only if it is awaitable. This matches the other provider handlers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
andrewklatzke
approved these changes
Sep 30, 2026
Merged
donei003
added a commit
that referenced
this pull request
Sep 30, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>launchdarkly-ai-server: 0.2.4</summary> ## [0.2.4](launchdarkly-ai-server-0.2.3...launchdarkly-ai-server-0.2.4) (2026-09-30) ### Features * **AIC-3210:** support graph().stream() in the Python AI SDK ([#104](#104)) ([25a345a](25a345a)) * **AIC-3211:** record each graph node as its own tracking event ([#114](#114)) ([c3c439e](c3c439e)) * **AIC-3449:** refactor AI Config reference into a dataclass ([#113](#113)) ([3ac06b4](3ac06b4)) * **AIC-3449:** Support pulling in configs for evals from code ([#108](#108)) ([fa374e2](fa374e2)) * **evaluations:** preserve the tool trajectory for judges ([#89](#89)) ([bcaa8c8](bcaa8c8)) ### Bug Fixes * **graph:** rename the graph span back to launchdarkly.graph, thread trajectory into stream judges ([79c9eeb](79c9eeb)) * **judges:** bound recorded values in memory, and show config's full tool catalog ([25fafef](25fafef)) * **judges:** exclude native tools from the config's available-tool list ([245a0a5](245a0a5)) * **judges:** keep trajectory recording total and observational ([57386b6](57386b6)) </details> <details><summary>launchdarkly-ai-python: 0.1.8</summary> ## [0.1.8](launchdarkly-ai-python-0.1.7...launchdarkly-ai-python-0.1.8) (2026-09-30) ### Features * **evaluations:** preserve the tool trajectory for judges ([#89](#89)) ([bcaa8c8](bcaa8c8)) </details> <details><summary>launchdarkly-ai-claude-agents: 0.2.4</summary> ## [0.2.4](launchdarkly-ai-claude-agents-0.2.3...launchdarkly-ai-claude-agents-0.2.4) (2026-09-30) ### Features * **AIC-3211:** record each graph node as its own tracking event ([#114](#114)) ([c3c439e](c3c439e)) </details> <details><summary>launchdarkly-ai-openai-agents: 0.2.4</summary> ## [0.2.4](launchdarkly-ai-openai-agents-0.2.3...launchdarkly-ai-openai-agents-0.2.4) (2026-09-30) ### Features * **AIC-3211:** record each graph node as its own tracking event ([#114](#114)) ([c3c439e](c3c439e)) ### Bug Fixes * **openai-agents:** accept sync tool handlers ([0ac9231](0ac9231)) * **openai-agents:** accept sync tool handlers ([#119](#119)) ([656d922](656d922)) </details> <details><summary>launchdarkly-ai-langchain-agents: 0.2.4</summary> ## [0.2.4](launchdarkly-ai-langchain-agents-0.2.3...launchdarkly-ai-langchain-agents-0.2.4) (2026-09-30) ### Features * **AIC-3211:** record each graph node as its own tracking event ([#114](#114)) ([c3c439e](c3c439e)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Release Please** bumps monorepo package versions and adds changelog entries; there is no new application source in the diff—only `.release-please-manifest.json`, `pyproject.toml`, `__version__`, and `CHANGELOG.md` updates. > > **`launchdarkly-ai-server` 0.2.4** (and aligned **claude / openai / langchain-agents** 0.2.4, **`launchdarkly-ai-python`** 0.1.8) documents already-merged work: **`graph().stream()`**, **per-graph-node tracking events**, **AI Config as a dataclass** and **eval configs from code**, **tool trajectory preserved for judges**, plus judge/graph telemetry fixes (e.g. `launchdarkly.graph` span, bounded trajectory memory, native tools excluded from available-tool lists) and **openai-agents** accepting **sync tool handlers**. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 27fba0e. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
create_openai_agent_handler()runsawait handler(args)for every tool call. When a tool handler is a sync function, the result is a plain value. Theawaitthen raisesTypeError: object dict can't be used in 'await' expression, and the agent run fails.This change calls the tool handler and awaits the result only when it is awaitable. The OpenAI messages, Claude messages, and Claude agents handlers already accept sync tool handlers. The OpenAI agent handler now does the same.
Why this appears in offline evaluations only
wrap_tool_handlersintracking.pywraps each tool in anasyncwrapper. The handler always gets a coroutine function, so the unconditionalawaitworks.EvaluationsModule.rungives the handler the tools fromTrajectoryRecorder.wrap. That wrapper keeps a sync tool sync on purpose (seeTrajectoryRecorder._record). The unconditionalawaitthen fails.The handler is the correct place for the fix. The tool type is
Callable[..., Any], which allows sync tools, and a handler that someone calls directly with sync tools fails in the same way.inspect.isawaitableon the result also covers a sync wrapper that returns a coroutine, which the offline recorder can return.Testing
packages/openai-agents/tests/test_handler.pycover a sync tool, an async tool, and a sync wrapper that returns an awaitable. Without the fix, the sync-tool test fails with theTypeError.uv run pytest packages/openai-agents: 130 passed.ruff check,ruff format --check, andmypy packages/openai-agents/srcpass.create_openai_agent_handler(), a synclookup_ordertool, and a{{message_history}}trajectory judge. Before the fix, the row errored withgeneration did not complete. After the fix, the row passed.🤖 Generated with Claude Code
Note
Overview
Fixes tool dispatch in
create_openai_agent_handler()so_build_agent_toolsno longer alwaysawaits the registered handler. It now invokeshandler(args)and awaits only wheninspect.isawaitableis true, matching other provider handlers and fixingTypeErrorwhen offline evaluations pass sync tools (e.g. from trajectory recording) instead of async-wrapped online tools.Adds unit tests covering plain sync handlers, async handlers, and sync functions that return a coroutine.
Reviewed by Cursor Bugbot for commit 0ac9231. Bugbot is set up for automated code reviews on this repo. Configure here.