Skip to content

fix(tools): report bad tool arguments as a sentence, not a JSON dump - #2559

Merged
jonathanKingston merged 3 commits into
mainfrom
claude/validate-outstanding-issues-bdhpr7
Sep 9, 2026
Merged

jonathanKingston merged 3 commits into
mainfrom
claude/validate-outstanding-issues-bdhpr7

Conversation

@claude

@claude claude Bot commented Sep 7, 2026

Copy link
Copy Markdown

The defect

A ZodError's message is its pretty-printed issues array. ToolRegistry.execute let one escape from tool.parse(rawArgs) (tool-registry.ts:139), and the agent loop splices whatever it catches into Error: ${msg} as the tool result (run-agent-loop.ts:949). So a model that gets one field wrong prints this into the chat:

Error: [
  {
    "expected": "string",
    "code": "invalid_type",
    "path": [ "todos", 0, "content" ],
    "message": "Invalid input: expected string, received undefined"
  }
]

Both audiences are worse off. The user reads a wall of JSON with the real problem buried in it. The model reads a payload to parse rather than an instruction it can act on, so its retry is a guess.

The change

Restate argument-validation failures at the registry — the one place that knows the arguments, rather than the tool's own execution, were at fault:

update_todos: the arguments did not match the tool's schema. todos[0].content — expected string, received undefined. Correct them and call the tool again.

  • paths render as todos[0].content, matching the shape of the call the model just made, so the field is findable without translating a JSON pointer
  • zod's Invalid input: prefix is dropped, since the surrounding sentence already says it
  • at most five fields are listed, then and N more — past a handful the model needs to re-read the schema, not a longer list
  • errors thrown from inside a tool are untouched; describeToolArgError returns null for anything that isn't a ZodError

This helps every tool, not just update_todos — the same escape existed for all of them.

Scope

The issue also asks for tolerance of near-miss todo shapes. I have not done that, deliberately: the screenshot captures the error but not the arguments, so I would be guessing at what qwen actually sent (title for content? a bare string?). Inventing aliases on a guess is how you get a schema nobody can reason about.

A readable message plausibly closes most of the gap on its own, since it tells the model exactly which field to fix. If it recurs, the captured payload is what's needed to design tolerance properly — worth reopening with one attached.

Testing

  • tool-arg-error.test.ts — the reported missing-content plan, the dropped prefix, the five-field cap, a wholly wrong argument object, and execution errors passing through untouched
  • Two tests in tool-registry.test.ts covering the path end to end, including that the tool does not run when validation fails
  • Verified the registry test discriminates: reverting the guard fails it and leaves the other 19 passing
  • Full unit suite 8653 pass / 0 fail / 3 skipped; typecheck, lint and oxfmt clean

Fixes #2442


🤖 Generated with Claude Code

https://claude.ai/code/session_012AZp6uiHQb3WbsHWyQqG76


Generated by Claude Code

claude and others added 2 commits September 7, 2026 18:36
A `ZodError`'s `message` is its pretty-printed `issues` array, and
`ToolRegistry.execute` let one escape from `tool.parse(rawArgs)` straight
into the agent loop's `Error: ${msg}` tool result. A model that got one
field wrong therefore printed forty lines of `{"expected": "string",
"code": "invalid_type", "path": ["todos", 0, "content"]}` into the chat.

Both audiences are worse off for it: the user reads a wall of JSON with
the actual problem buried in it, and the model reads a payload to parse
instead of an instruction it can act on, so its retry is a guess.

Restate argument-validation failures at the registry, which is the one
place that knows the arguments — rather than the tool's own execution —
were at fault:

    update_todos: the arguments did not match the tool's schema.
    todos[0].content — expected string, received undefined.
    Correct them and call the tool again.

Paths render as `todos[0].content` so the field is findable in the call
the model just made; zod's redundant "Invalid input:" prefix is dropped;
the list caps at five fields and then counts the rest. Errors thrown from
inside a tool are untouched — only a `ZodError` is restated.

Reported against a qwen3-235b plan whose first todo carried no `content`.

Fixes #2442

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AZp6uiHQb3WbsHWyQqG76
@jonathanKingston
jonathanKingston marked this pull request as ready for review September 9, 2026 01:27
github-actions Bot added a commit that referenced this pull request Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

🖥️ PR preview

github-actions Bot added a commit that referenced this pull request Sep 9, 2026
@jonathanKingston
jonathanKingston merged commit 9a57ad4 into main Sep 9, 2026
24 checks passed
@jonathanKingston
jonathanKingston deleted the claude/validate-outstanding-issues-bdhpr7 branch September 9, 2026 01:52
github-actions Bot added a commit that referenced this pull request Sep 9, 2026

This branch was previously deployed

1 inactive deployment
github-pages — 4e4a494a Deployed Sep 9, 2026 by github-actions[bot] via deploy #122
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Malformed todo updates surface a raw Zod error in the transcript

2 participants