Skip to content

fix(langgraph): offload sync remote tool coroutines - #229

Open
fashkl wants to merge 4 commits into
oracle:mainfrom
fashkl:fix/offload-sync-langgraph-tools
Open

fashkl wants to merge 4 commits into
oracle:mainfrom
fashkl:fix/offload-sync-langgraph-tools

Conversation

@fashkl

@fashkl fashkl commented Aug 16, 2026

Copy link
Copy Markdown

Problem

Fixes #228.

LangGraph RemoteTool conversion exposes a coroutine for sync remote-tool callables, but the coroutine currently calls the synchronous function directly. For RemoteTool, that function executes httpx.request(...), so async LangGraph execution can block the event loop while waiting on network I/O.

This is worse than leaving StructuredTool.coroutine unset. LangChain's own StructuredTool._arun checks self.coroutine first; when it is None, it falls back to the base async implementation, whose source comment says it is expected to delegate _run to a separate thread. By supplying a fake async wrapper, pyagentspec bypasses that safe default and runs blocking sync I/O on the event-loop thread.

This also makes RemoteTool inconsistent with the existing ApiNode executor split in this codebase: ApiNodeExecutor._execute() uses sync httpx.request(...), while ApiNodeExecutor._aexecute() uses httpx.AsyncClient. RemoteTool is the outlier because its async entrypoint is only async-shaped, not async-safe.

In services with event-loop based liveness/readiness handling, a slow or stuck remote tool request can therefore freeze unrelated async work long enough to trip health checks and restart the service.

Fix

Offload synchronous structured-tool coroutine execution with asyncio.to_thread(...) while preserving native async callables unchanged.

This is a small general safety net for sync callables that pyagentspec intentionally exposes through a coroutine, including the current RemoteTool and ClientTool conversion paths. A future native async RemoteTool implementation using httpx.AsyncClient would still be compatible with this change and could further reduce thread usage for RemoteTool specifically.

Testing

  • Added test_remote_tool_coroutine_does_not_block_event_loop
  • Ran SKIP_LLM_TESTS=1 uv run python -m pytest -q tests/adapters/langgraph/test_tools.py
  • Ran black and isort checks for the changed files

Commit is signed off per the contribution guide.

Signed-off-by: Mohamed Ali <mohamed.ali@openinnovation.ai>
@fashkl
fashkl requested a review from a team August 16, 2026 19:19
@oracle-contributor-agreement

Copy link
Copy Markdown

Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA).
The following contributors of this PR have not signed the OCA:

To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application.

When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated.

If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public.

@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Required At least one contributor does not have an approved Oracle Contributor Agreement. label Aug 16, 2026
@cesarebernardis

Copy link
Copy Markdown
Member

@fashkl Thank you for the contribution!

To start, could you please sign the Oracle Contributor Agreement (OCA)?

You can find more info in the comment at #229 (comment). Please make sure to use the same email associated to your Github account, that should be the one reported in the comment.

@fashkl

fashkl commented Aug 31, 2026

Copy link
Copy Markdown
Author

@fashkl Thank you for the contribution!

To start, could you please sign the Oracle Contributor Agreement (OCA)?

You can find more info in the comment at #229 (comment). Please make sure to use the same email associated to your Github account, that should be the one reported in the comment.

Hello @cesarebernardis
Already I signed the OCA and the request is pending more than two weeks.

@fede-kamel

Copy link
Copy Markdown
Member

Reviewed and verified this locally against current main (it merges cleanly, 2 commits behind):

  • The regression test fails without the asyncio.to_thread change (assert (t1 - t0) < 0.1 trips because the blocking httpx.request runs on the loop) and passes with it.
  • tests/adapters/langgraph/test_tools.py as a whole: 37 passed with the change applied.
  • asyncio.to_thread also propagates contextvars, so LangChain callback context survives the hop, and native coroutines still bypass the wrapper via _is_async_callable. Same approach the reporter suggested in LangGraph RemoteTool coroutine blocks the event loop #228, and consistent with how ApiNodeExecutor already separates sync and async HTTP.

The only thing holding this back is the OCA check, and @fashkl reported on Aug 31 that they signed more than two weeks earlier. @cesarebernardis, could someone check with the OSS office why the approval is still pending? Once the bot flips this looks ready to me.

@fede-kamel fede-kamel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up verification focused on what changes when the sync callable moves to a worker thread (PR head 2f6f016, LangGraph 1.2.4, langchain-core 1.4.9):

Verified

  • tests/adapters/langgraph + tests/adapters/test_template_rendering.py on Python 3.12: 151 passed, 89 skipped (LLM tests). The new test passes 10/10 consecutive runs.
  • Nested interrupts through the thread hop: a ClientTool with requires_confirmation=True run with ainvoke yields the confirmation interrupt (action_requests/review_configs), resuming with approve yields the client_tool_request interrupt with the right kwargs, resuming the value completes the flow. Resuming with reject propagates the adapter's RuntimeError("Tool ... was denied by the user") out of to_thread as before. So interrupt() inside _confirm_then and inside client_tool both work from the worker thread (context is copied by asyncio.to_thread).
  • Tracing context: get_current_span() inside a remote-tool callable during ainvoke returns the ToolNodeExecution[...] span both on main and on this branch, so the change does not alter which span the callable sees. (That the ToolExecution span opened by AgentSpecToolCallbackHandler.on_tool_start is not the current span inside the callable in async runs is pre-existing and unrelated to this PR.)
  • Equivalence with the default: asyncio.to_thread does the same thing LangChain's own _arun fallback does for func-only tools (run_in_executor(None, ...) wrapping the call in copy_context().run on the loop's default executor), which is also what LangGraph uses for sync nodes in async runs. Only RemoteTool (_langgraphconverter.py:807) and ClientTool (:943) go through _as_structured_tool_coroutine; sync ServerTool callables already used the LangChain fallback, so this brings the two paths in line.
  • Python 3.10 (LangGraph extra installed, which the public CI does not do): main fails tests/adapters/langgraph/flows/test_toolnode.py::test_toolnode_can_be_executed_async_with_interrupt_resume with RuntimeError: Called get_config outside of a runnable context (3/3 runs), this branch passes it (3/3 runs). That is exactly the situation the load-time warning at _langgraphconverter.py:899 describes, so the thread hop also fixes async client-tool interrupts on 3.10 in my runs.

Suggestions (non-blocking)

  • test_remote_tool_coroutine_does_not_block_event_loop asserts time.monotonic() - started_at < 0.1 after a 0.01 s sleep while the mock blocks 0.2 s. It is stable locally, but on a slow runner a wider margin (e.g. block 0.5 s, assert < 0.25 s, keep not task.done()) would make the intent survive scheduling jitter.
  • Given the 3.10 result above, it may be worth checking whether the < (3, 11) warning for client tools can be narrowed once this lands; happy to do that as a follow-up if useful.

Replace the wall-clock threshold in
test_remote_tool_coroutine_does_not_block_event_loop with threading.Event
gating, so the test no longer depends on scheduling jitter on slow runners.

The mock signals when the blocking request is entered and waits on a bounded
release event. The test observes that signal from the event loop and asserts
the task has not completed, which is the actual property under test. A
regression still fails (on `not task.done()`) instead of hanging.
@fashkl

fashkl commented Sep 14, 2026

Copy link
Copy Markdown
Author

Applied the test suggestion in 2759730, though I changed it a bit more than you proposed. Instead of widening the window I dropped the wall-clock assert entirely: the mock sets a threading.Event when it's entered, then waits on a release event with a 5s timeout. The test polls that first event from the loop, asserts not task.done(), and releases the worker in a finally. That's the property we actually care about and it doesn't depend on runner speed. It also took the test from ~0.21s to ~0.08s.

It still catches the regression: with _as_structured_tool_coroutine put back to return func(*args, **kwargs) it fails on assert not task.done() after about 5s (the bounded wait), instead of hanging.

My runs, Python 3.13.13 / langgraph 1.2.11 / langchain-core 1.4.9:

  • test_tools.py: 34 passed, 3 skipped. New test 10/10.
  • tests/adapters/langgraph + test_template_rendering.py: 148 passed, 86 skipped, 3 failed, 3 errors. All six also fail with the asyncio.to_thread change reverted, so they're pre-existing: test_ocigenai_conversion.py::test_reverse_convert_chatocigenai_to_agentspec plus the MCP TLS and MCP server-fixture ones.
  • Confirmed your equivalence point too. langchain_core/tools/base.py:900 calls run_in_executor(None, self._run, ...), which resolves to loop.run_in_executor(None, partial(copy_context().run, wrapper)) in runnables/config.py:705-710. Same thing asyncio.to_thread does.

On the < (3, 11) warning, I'd rather leave it for the follow-up you offered. I can't reproduce your 3.10 result here (no 3.10 interpreter with the langgraph extra, and CI doesn't install it either), so it would be resting on your runs alone, and if we're wrong users just get the bare Called get_config outside of a runnable context with nothing to explain it. The warning also covers interrupt() in _confirm_tool_use, not only the client_tool path this PR moved, so narrowing it probably wants 3.10 coverage in CI first.

OCA is still the only red check. @cesarebernardis, anything from the OSS office? I signed in mid-August.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Required At least one contributor does not have an approved Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LangGraph RemoteTool coroutine blocks the event loop

3 participants