Skip to content

fix(openai-agents): accept sync tool handlers - #119

Merged
donei003 merged 1 commit into
mainfrom
fix/openai-agents-sync-tool-handlers
Sep 30, 2026
Merged

donei003 merged 1 commit into
mainfrom
fix/openai-agents-sync-tool-handlers

Conversation

@donei003

@donei003 donei003 commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

create_openai_agent_handler() runs await handler(args) for every tool call. When a tool handler is a sync function, the result is a plain value. The await then raises TypeError: object dict can't be used in 'await' expression, and the agent run fails.

This change calls the tool handler and awaits the result only when it is awaitable. The OpenAI messages, Claude messages, and Claude agents handlers already accept sync tool handlers. The OpenAI agent handler now does the same.

Why this appears in offline evaluations only

  • Online path: wrap_tool_handlers in tracking.py wraps each tool in an async wrapper. The handler always gets a coroutine function, so the unconditional await works.
  • Offline path: EvaluationsModule.run gives the handler the tools from TrajectoryRecorder.wrap. That wrapper keeps a sync tool sync on purpose (see TrajectoryRecorder._record). The unconditional await then fails.

The handler is the correct place for the fix. The tool type is Callable[..., Any], which allows sync tools, and a handler that someone calls directly with sync tools fails in the same way. inspect.isawaitable on the result also covers a sync wrapper that returns a coroutine, which the offline recorder can return.

Testing

  • New unit tests in packages/openai-agents/tests/test_handler.py cover a sync tool, an async tool, and a sync wrapper that returns an awaitable. Without the fix, the sync-tool test fails with the TypeError.
  • uv run pytest packages/openai-agents: 130 passed.
  • ruff check, ruff format --check, and mypy packages/openai-agents/src pass.
  • Live check: I ran an offline evaluation with only create_openai_agent_handler(), a sync lookup_order tool, and a {{message_history}} trajectory judge. Before the fix, the row errored with generation did not complete. After the fix, the row passed.

🤖 Generated with Claude Code


Note

Overview
Fixes tool dispatch in create_openai_agent_handler() so _build_agent_tools no longer always awaits the registered handler. It now invokes handler(args) and awaits only when inspect.isawaitable is true, matching other provider handlers and fixing TypeError when offline evaluations pass sync tools (e.g. from trajectory recording) instead of async-wrapped online tools.

Adds unit tests covering plain sync handlers, async handlers, and sync functions that return a coroutine.

Reviewed by Cursor Bugbot for commit 0ac9231. Bugbot is set up for automated code reviews on this repo. Configure here.

The OpenAI agent handler awaited every tool handler result. A sync tool
handler returns a plain value, so the call raised a TypeError and the
agent run failed.

Call the tool handler, then await the result only if it is awaitable.
This matches the other provider handlers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

@donei003
donei003 merged commit 656d922 into main Sep 30, 2026
9 checks passed
@donei003
donei003 deleted the fix/openai-agents-sync-tool-handlers branch September 30, 2026 18:45
@github-actions github-actions Bot mentioned this pull request Sep 29, 2026
donei003 added a commit that referenced this pull request Sep 30, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>launchdarkly-ai-server: 0.2.4</summary>

##
[0.2.4](launchdarkly-ai-server-0.2.3...launchdarkly-ai-server-0.2.4)
(2026-09-30)


### Features

* **AIC-3210:** support graph().stream() in the Python AI SDK
([#104](#104))
([25a345a](25a345a))
* **AIC-3211:** record each graph node as its own tracking event
([#114](#114))
([c3c439e](c3c439e))
* **AIC-3449:** refactor AI Config reference into a dataclass
([#113](#113))
([3ac06b4](3ac06b4))
* **AIC-3449:** Support pulling in configs for evals from code
([#108](#108))
([fa374e2](fa374e2))
* **evaluations:** preserve the tool trajectory for judges
([#89](#89))
([bcaa8c8](bcaa8c8))


### Bug Fixes

* **graph:** rename the graph span back to launchdarkly.graph, thread
trajectory into stream judges
([79c9eeb](79c9eeb))
* **judges:** bound recorded values in memory, and show config's full
tool catalog
([25fafef](25fafef))
* **judges:** exclude native tools from the config's available-tool list
([245a0a5](245a0a5))
* **judges:** keep trajectory recording total and observational
([57386b6](57386b6))
</details>

<details><summary>launchdarkly-ai-python: 0.1.8</summary>

##
[0.1.8](launchdarkly-ai-python-0.1.7...launchdarkly-ai-python-0.1.8)
(2026-09-30)


### Features

* **evaluations:** preserve the tool trajectory for judges
([#89](#89))
([bcaa8c8](bcaa8c8))
</details>

<details><summary>launchdarkly-ai-claude-agents: 0.2.4</summary>

##
[0.2.4](launchdarkly-ai-claude-agents-0.2.3...launchdarkly-ai-claude-agents-0.2.4)
(2026-09-30)


### Features

* **AIC-3211:** record each graph node as its own tracking event
([#114](#114))
([c3c439e](c3c439e))
</details>

<details><summary>launchdarkly-ai-openai-agents: 0.2.4</summary>

##
[0.2.4](launchdarkly-ai-openai-agents-0.2.3...launchdarkly-ai-openai-agents-0.2.4)
(2026-09-30)


### Features

* **AIC-3211:** record each graph node as its own tracking event
([#114](#114))
([c3c439e](c3c439e))


### Bug Fixes

* **openai-agents:** accept sync tool handlers
([0ac9231](0ac9231))
* **openai-agents:** accept sync tool handlers
([#119](#119))
([656d922](656d922))
</details>

<details><summary>launchdarkly-ai-langchain-agents: 0.2.4</summary>

##
[0.2.4](launchdarkly-ai-langchain-agents-0.2.3...launchdarkly-ai-langchain-agents-0.2.4)
(2026-09-30)


### Features

* **AIC-3211:** record each graph node as its own tracking event
([#114](#114))
([c3c439e](c3c439e))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> **Release Please** bumps monorepo package versions and adds changelog
entries; there is no new application source in the diff—only
`.release-please-manifest.json`, `pyproject.toml`, `__version__`, and
`CHANGELOG.md` updates.
> 
> **`launchdarkly-ai-server` 0.2.4** (and aligned **claude / openai /
langchain-agents** 0.2.4, **`launchdarkly-ai-python`** 0.1.8) documents
already-merged work: **`graph().stream()`**, **per-graph-node tracking
events**, **AI Config as a dataclass** and **eval configs from code**,
**tool trajectory preserved for judges**, plus judge/graph telemetry
fixes (e.g. `launchdarkly.graph` span, bounded trajectory memory, native
tools excluded from available-tool lists) and **openai-agents**
accepting **sync tool handlers**.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
27fba0e. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants