feat: add auto approval and auto preset selection

.changeset/eve-extension-auto-modes.md

@@ -0,0 +1,12 @@
+---
+"@github-tools/eve-extension": minor
+"@github-tools/sdk": minor
+---
+
+Add `'auto'` modes to the eve extension. They need `ai` 7.0.105 or later.
+
+- `preset: 'auto'` routes each user message to at most two presets with the evaluation model and registers only their tools for that turn. Tools the agent already called stay registered, so a parked approval can still resume after routing changes.
+- `requireApproval: 'auto'` lets low-risk write tools (`AUTO_APPROVAL_TOOLS`) run without a prompt when the evaluation model rates the call low-risk and the user's latest request asked for it. Other write tools keep requiring approval. Per-tool values in `requireApproval` and `overrides[tool].approval` also accept `'auto'`.
+- The `evaluation` option (`{ model, maxRisk, minIntent, minPresetProbability, maxPresets }`) tunes both modes, as in the SDK.
+
+`@github-tools/sdk/eve-runtime` now exports `AUTO_APPROVAL_TOOLS`, `latestUserText`, `needsAutoApproval`, `selectPresets` and the `GithubEvaluationOptions` type.

.changeset/eve-structured-error-output.md

@@ -0,0 +1,5 @@
+---
+"@github-tools/sdk": patch
+---
+
+Fix eve tool output formatting for GitHub API errors. A 404, 403 or rate-limit error from a tool with a built-in eve formatter (`listPullRequestFiles`, `getFileContent`, `getRepositoryTree`, `getPullRequestContext`, `getCommit`, `compareCommits`) no longer breaks the next model step with `Cannot read properties of undefined`. The structured error now reaches the model as is.

.changeset/sdk-auto-approval-presets.md

@@ -0,0 +1,13 @@
+---
+"@github-tools/sdk": minor
+---
+
+Add `'auto'` modes for approval and presets. Both need `ai` 7.0.105 or later. Everything else still works on `ai` 6.
+
+- `requireApproval: 'auto'` on `createGithubTools` / `createGithubAgent`: low-risk write tools (`AUTO_APPROVAL_TOOLS`: labels, assignees, reactions, comments, review-thread replies, reviewer requests, notification reads, workflow re-runs) run without a prompt when an evaluation model rates the call low-risk and the latest user message asked for it. Otherwise approval is requested as usual. Other write tools keep requiring approval. Per-tool values also accept `'auto'`, for example `requireApproval: { updateIssue: 'auto', mergePullRequest: true }`.
+- `createGithubAgent({ preset: 'auto' })` selects presets per call from the latest user message, narrows the active tools, and uses the matching preset's system prompt. It combines at most two presets. Read-only `repo-explorer` is only used when no other preset matches. When none clears the threshold it uses the most likely one. It never exposes the full catalog.
+- The new optional `evaluation` option (`{ model, maxRisk, minIntent, minPresetProbability, maxPresets }`, defaults `1`, `0.6`, `0.7`, `2`) tunes both modes. The model defaults to TypeSafe's Jev (`'typesafe-ai/jev'` through AI Gateway).
+- When the evaluation call fails (model not enabled on AI Gateway, no credits, outage), `'auto'` approval asks for approval and `preset: 'auto'` uses `repo-explorer`. Both log a `github_tools.EVALUATION_FAILED` warning with the gateway error as `cause`.
+- `ToolOptions.needsApproval` on individual tool factories now also accepts an AI SDK approval function.
+
+`createDurableGithubAgent` does not accept `'auto'`.

AGENTS.md

@@ -77,6 +77,7 @@ export const myTool = (token: GithubTokenInput, { needsApproval = true }: ToolOp
 - `src/index.ts` — public API: `createGithubTools()`, `allTools` composition, re-exports
 - `src/agents.ts` — `createGithubAgent()` (`ToolLoopAgent`) with preset-specific system prompts
 - `src/workflow.ts` — `createDurableGithubAgent()` (`WorkflowAgent` from `@ai-sdk/workflow`), exported from `@github-tools/sdk/workflow` subpath
+- `src/core/evaluation.ts` — `requireApproval: 'auto'` and `preset: 'auto'` on AI SDK `experimental_evaluate` (default model `typesafe-ai/jev`). `ai` is a static namespace import and `experimental_evaluate` is feature-detected, so the root entry still loads on `ai` 6. Keep it static: a dynamic `import('ai')` splits the chunk and breaks `eve build` of the extension
 - `src/eve-runtime.ts` — shared eve primitives for `@github-tools/eve-extension` (`listEveToolDescriptors`, `executeGithubEveTool`, approval helpers); public export `@github-tools/sdk/eve-runtime`
 - `src/eve.ts` — deprecated consumer `createGithubTools` / per-tool factories for `agent/tools/` (`@github-tools/sdk/eve`)
 - `src/client.ts` — `createOctokit(token)` wrapper
@@ -100,7 +101,7 @@ Ten presets (`code-review`, `issue-triage`, `repo-explorer`, `ci-ops`, `security
 
 ## eve extension durable callbacks and schemas (`packages/github-tools-eve-extension`)
 
-eve rejects a dynamic tool whose callback or live validation schema has no durable descriptor. It stamps one per callback phase (`execute`, `toModelOutput`, `approval` / `approvalRequest`, `approvalKey`, and the `label` callbacks), and `execute` is the only required callback. In `extension/tools/github.ts`, every authored callback must be a **direct** `defineTool` property with an inline function (or identifier). Input and output schemas from the SDK or extension config must use `defineDurableSchema`. Conditional spreads and call expressions (`resolveEveApproval(...)`, `always()`) are invisible to eve's callback transform. Durable callbacks and schema factories may only close over a serializable tool `name` and re-read config via `buildSessionOptions()`. CI enforces this via `test/durable-define-tool.test.ts`. A scheduled canary (`.github/workflows/eve-canary.yml`) additionally builds the consumer example against a coherent `eve@latest` / `ai@latest` pair and inspects the durable schema transform.
+eve rejects a dynamic tool whose callback or live validation schema has no durable descriptor. It stamps one per callback phase (`execute`, `toModelOutput`, `approval` / `approvalRequest`, `approvalKey`, and the `label` callbacks), and `execute` is the only required callback. In `extension/tools/github.ts`, every authored callback must be a **direct** `defineTool` property with an inline function (or identifier). Input and output schemas from the SDK or extension config must use `defineDurableSchema`. Conditional spreads and call expressions (`resolveEveApproval(...)`, `always()`) are invisible to eve's callback transform. Durable callbacks and schema factories may only close over serializable values (the tool `name`, and the latest user request string for `approval`) and re-read config via `buildSessionOptions()` or `extension.config`. CI enforces this via `test/durable-define-tool.test.ts`. A scheduled canary (`.github/workflows/eve-canary.yml`) additionally builds the consumer example against a coherent `eve@latest` / `ai@latest` pair and inspects the durable schema transform.
 
 ## Chat App Architecture (`apps/chat`)
 

apps/docs/content/docs/2.frameworks/1.eve-extension.md

@@ -180,18 +180,36 @@ export default githubExtension({
 })
 ```
 
+## Automatic presets and approval
+
+For an agent that takes open-ended requests, let the extension pick the tools and decide which writes need a human. Both modes need `ai` 7.0.105 or later.
+
+```ts [agent/extensions/github.ts]
+import githubExtension from '@github-tools/eve-extension'
+
+export default githubExtension({
+  preset: 'auto',
+  requireApproval: 'auto',
+})
+```
+
+- `preset: 'auto'` asks the evaluation model which presets the latest user message needs and registers only their tools for that turn. It combines at most two presets and never exposes the full catalog. Tools the agent already called in the session stay registered, so a parked approval resumes even when routing changes.
+- `requireApproval: 'auto'` lets low-risk write tools (`AUTO_APPROVAL_TOOLS`: labels, assignees, reactions, comments, review-thread replies, reviewer requests, notification reads, workflow re-runs) run without a prompt when the model rates the call low-risk and the user's request asked for it. Other write tools keep `always()`. Mix it per tool: `requireApproval: { addLabels: 'auto', mergePullRequest: true }`.
+- `evaluation` tunes both: `minPresetProbability` and `maxPresets` for routing, `maxRisk` and `minIntent` for approval, and `model` to replace the default evaluation model.
+
 ## Config schema
 
 | Field | Type | Notes |
 |---|---|---|
 | `token` | `string \| (() => Promise<string>)` | PAT string, or an async provider for rotating tokens (e.g. a GitHub App installation token) — the same `GithubTokenInput` the SDK accepts; falls back to `GITHUB_TOKEN` when omitted and `connector` is not set |
 | `connector` | `string \| (() => string \| Promise<string>)` | Vercel Connect connector name, or a resolver to pick one dynamically (e.g. per environment/tenant); takes priority over `token` |
 | `connect` | `record \| ((ctx, call) => record)` | Passed through to `getToken` when `connector` is set; `connect.subject` defaults to `{ type: 'app' }` and also accepts a per-caller resolver, see [Per-user tokens](#per-user-tokens). A resolver picks params per tool call, see [Multiple GitHub App installations](#multiple-github-app-installations) |
-| `preset` | preset name or array | `code-review`, `issue-triage`, `ci-ops`, `repo-explorer`, `security-audit`, `release-manager`, `discussion-moderator`, `notification-inbox`, `pr-author`, `maintainer`, see [Presets](/guide/presets) |
+| `preset` | preset name, array, or `'auto'` | `code-review`, `issue-triage`, `ci-ops`, `repo-explorer`, `security-audit`, `release-manager`, `discussion-moderator`, `notification-inbox`, `pr-author`, `maintainer`, see [Presets](/guide/presets). `'auto'` routes each user message, see [Automatic presets and approval](#automatic-presets-and-approval) |
 | `include` | `string[]?` | Tool names to add on top of `preset` (union), or the full set standalone, see [Pick exact tools](#pick-exact-tools) |
 | `exclude` | `string[]?` | Tool names to remove from the resolved `preset` + `include` set |
 | `context` | `{ owner?, repo?, pullNumber?, issueNumber?, ref? }?` | Default owner/repo/number/ref for tool inputs — matching fields become optional and fill from context when omitted, see [Working context](/guide/working-context) |
-| `requireApproval` | `boolean \| record` | Global or per-tool; per-tool values may be `'once'`, `'always'`, `'never'`, or predicate functions |
+| `requireApproval` | `boolean \| 'auto' \| record` | Global or per-tool; per-tool values may be `'once'`, `'always'`, `'never'`, `'auto'`, or predicate functions |
+| `evaluation` | `{ model?, maxRisk?, minIntent?, minPresetProbability?, maxPresets? }?` | Tunes the `'auto'` modes, see [Automatic presets and approval](#automatic-presets-and-approval) |
 | `overrides` | `record` | Per-tool `description` / `approval` / `toModelOutput` / `outputSchema` |
 | `author` / `committer` / `coAuthors` | commit identity | Attribution for commit-creating tools, see [Commit Attribution](/guide/commit-attribution) |
 
@@ -208,7 +226,7 @@ export default githubExtension({
 
 ## Durable multi-turn sessions
 
-The extension registers each tool with an **authored inline** `execute`, `toModelOutput`, and `approval` as **direct** `defineTool` properties that only close over a serializable tool `name`, then rebuilds session options from the extension config on every call via `@github-tools/sdk/eve-runtime`. Input schemas and optional `overrides.outputSchema` values use `defineDurableSchema` with the same name-only closure, preserving Zod refinements and transformations across Workflow replay. Tools resolve on `step.started` so registration stays fresh across durable steps. That pattern survives multi-turn eve Workflow replay (see [#51](https://github.com/vercel-labs/github-tools/issues/51), [#99](https://github.com/vercel-labs/github-tools/issues/99)). A spread or call-expression callback (`resolveEveApproval(...)`, a ternary-wrapped `toModelOutput`) has no durable descriptor, and eve rejects a dynamic tool whose callback is missing one. Prefer this mount over the deprecated [`createGithubTools`](/deprecated/eve) / [`connectGithubTools`](/deprecated/eve) paths for Slack / multi-turn durable agents — those register tools from inside `node_modules` and are skipped on replay. Author `overrides.toModelOutput` inline in the agent; a function imported from a library will not get a durable descriptor.
+The extension registers each tool with an **authored inline** `execute`, `toModelOutput`, and `approval` as **direct** `defineTool` properties that only close over serializable values (the tool `name`, and for `approval` the latest user request text), then rebuilds session options from the extension config on every call via `@github-tools/sdk/eve-runtime`. Input schemas and optional `overrides.outputSchema` values use `defineDurableSchema` with the same name-only closure, preserving Zod refinements and transformations across Workflow replay. Tools resolve on `step.started` so registration stays fresh across durable steps. That pattern survives multi-turn eve Workflow replay (see [#51](https://github.com/vercel-labs/github-tools/issues/51), [#99](https://github.com/vercel-labs/github-tools/issues/99)). A spread or call-expression callback (`resolveEveApproval(...)`, a ternary-wrapped `toModelOutput`) has no durable descriptor, and eve rejects a dynamic tool whose callback is missing one. Prefer this mount over the deprecated [`createGithubTools`](/deprecated/eve) / [`connectGithubTools`](/deprecated/eve) paths for Slack / multi-turn durable agents — those register tools from inside `node_modules` and are skipped on replay. Author `overrides.toModelOutput` inline in the agent; a function imported from a library will not get a durable descriptor.
 
 Object-shaped execute results include `rateLimit` (`remaining`, `limit`, `reset`, `resource`). `toModelOutput` strips it so the model never sees the remaining count; `toolResultFrom` and channels still do. See [Rate-limit metadata](/api/reference#rate-limit-metadata).
 
@@ -224,6 +242,7 @@ Approval **pauses the session durably** until a human responds, and policies are
 | `false` / `'never'` | omit `approval` | Skip approval (eve default) |
 | `'once'` | `once()` | Approve once per session, then auto-allow |
 | predicate | custom `Approval` | Input-dependent gate (`toolInput`, session context) |
+| `'auto'` | evaluation model | Low-risk write tools run when the call matches the user's request; otherwise approval is requested |
 
 Default (no `requireApproval`): all write tools → `always()`. Unlisted write tools keep the `always()` fail-safe default. Read tools never require approval. Details: [Control write safety](/guide/approval-control).
 

apps/docs/content/docs/4.guide/1.presets.md

@@ -91,6 +91,30 @@ const tools = createGithubTools({
 })
 ```
 
+## Let the agent pick presets
+
+When one agent handles open-ended requests, use `preset: 'auto'`:
+
+```ts [auto-preset.ts]
+import { createGithubAgent } from '@github-tools/sdk'
+
+const agent = createGithubAgent({
+  model: 'anthropic/claude-opus-5.5',
+  preset: 'auto',
+})
+
+await agent.generate({ prompt: 'Why is CI failing on main?' })
+// runs with the ci-ops tools and system prompt
+```
+
+On each call, a fast evaluation model reads the latest user message and rates how likely it is to need each preset. The agent then exposes only the tools of the chosen presets, and uses the preset's system prompt when exactly one is chosen. The full catalog is never exposed:
+
+1. Presets rated at or above `0.7` are used, most likely first, up to two per call.
+2. The read-only `repo-explorer` preset is only used when no other preset qualifies, since every other preset already has the read tools its task needs.
+3. When none reaches `0.7`, the single most likely preset is used. When nothing stands out, as with "hello", that is `repo-explorer`.
+
+`preset: 'auto'` is available on `createGithubAgent` and needs `ai` 7.0.105 or later. To tune it, use `evaluation: { minPresetProbability, maxPresets, model }`. The default model is TypeSafe's Jev. If the evaluation call fails (model not enabled on AI Gateway, no credits, outage), the call uses read-only `repo-explorer` and logs a `github_tools.EVALUATION_FAILED` warning through `console.warn`.
+
 ## Pick the right preset
 
 | Preset | Tools included | Use case |

apps/docs/content/docs/4.guide/2.approval-control.md

@@ -27,7 +27,7 @@ actions:
   - windsurf
 ---
 
-Configure @github-tools/sdk write safety in this project: use requireApproval on createGithubTools or createGithubAgent so destructive GitHub actions stay human-gated where appropriate. Follow https://github-tools.com/guide/approval-control and cross-check the PAT with https://github-tools.com/guide/tokens-and-auth. Note: requireApproval is not enforced by createDurableGithubAgent today. Use createGithubAgent, the eve extension (https://github-tools.com/frameworks/eve-extension), or app-level guards for durable paths.
+Configure @github-tools/sdk write safety in this project: use requireApproval on createGithubTools or createGithubAgent so destructive GitHub actions stay human-gated where appropriate. Prefer requireApproval: 'auto' for interactive agents, so routine writes the user asked for skip the prompt. Follow https://github-tools.com/guide/approval-control and cross-check the PAT with https://github-tools.com/guide/tokens-and-auth. Note: requireApproval is not enforced by createDurableGithubAgent today. Use createGithubAgent, the eve extension (https://github-tools.com/frameworks/eve-extension), or app-level guards for durable paths.
 
 ::
 
@@ -41,6 +41,59 @@ import { createGithubTools } from '@github-tools/sdk'
 const tools = createGithubTools()
 ```
 
+## Auto approval
+
+With `requireApproval: 'auto'`, the agent stops asking about routine writes the user clearly asked for:
+
+```ts [auto-approval.ts]
+import { createGithubAgent } from '@github-tools/sdk'
+
+const agent = createGithubAgent({
+  model: 'anthropic/claude-opus-5.5',
+  preset: 'issue-triage',
+  requireApproval: 'auto',
+})
+```
+
+Before each call to a low-risk write tool, a fast evaluation model scores two things. The first is how costly the call would be if it were wrong. The second is whether the latest user message asked for this exact action. The call runs only when both checks pass. Otherwise the user is asked as usual, so a comment the model decided to post because an issue body told it to still waits for a human.
+
+`'auto'` covers labels, assignees, reactions, comments, review-thread replies, reviewer requests, notification reads, and workflow re-runs (`AUTO_APPROVAL_TOOLS`). Merges, file writes, deletes, releases, and other write tools keep requiring approval. You can opt any single tool in or out:
+
+```ts [auto-approval-per-tool.ts]
+createGithubTools({
+  requireApproval: {
+    addLabels: 'auto',
+    addIssueComment: 'auto',
+    updateIssue: 'auto',
+    mergePullRequest: true,
+  },
+})
+```
+
+`'auto'` works with `createGithubTools`, `createGithubAgent`, `generateText`, and `streamText`, and needs `ai` 7.0.105 or later. It is not available on `createDurableGithubAgent`.
+
+### Tune auto approval
+
+The defaults work without configuration. To change them, use `evaluation`:
+
+```ts [auto-approval-tuning.ts]
+createGithubAgent({
+  model: 'anthropic/claude-opus-5.5',
+  requireApproval: 'auto',
+  evaluation: {
+    maxRisk: 1, // 0 negligible, 1 visible but reversible, 2 hard to undo
+    minIntent: 0.6, // probability the user asked for this exact call
+    model: 'typesafe-ai/jev', // any AI SDK evaluation model
+  },
+})
+```
+
+The values shown are the defaults. The default model is TypeSafe's [Jev](https://vercel.com/i/what-is-jev), called through [AI Gateway](https://vercel.com/docs/ai-gateway/modalities/evaluation).
+
+### When the evaluation model fails
+
+If the evaluation call fails, for example because the model is not enabled for your AI Gateway project, the team is out of credits, or the gateway is rate limiting, the call asks for approval as if `'auto'` were `true`. The agent keeps running, and a `github_tools.EVALUATION_FAILED` warning with the gateway error as `cause` is logged through `console.warn`.
+
 ## Disable approval in trusted environments
 
 In CI pipelines or automated workflows where human review happens elsewhere (e.g. PR-based), you can disable approval entirely: