Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -824,6 +824,15 @@ changing the original error code, message, status, retryability, or failure
stage. Capture never triggers a provider call or changes retry behavior; the
caller owns retention and deletion of the local file.

The allowlist recognizes Claude's `api_error` terminal reason and the `success`
result subtype, including the documented case where a result is still marked
`is_error: true`. A structured `api_error` without a finite numeric API status
is classified as a retryable transient provider failure with fixed diagnostics
and a generic message. When a numeric status is present, status mapping takes
precedence, so authentication, quota, rate-limit, server-error, and other
status-bearing behavior remains unchanged. Dynamic provider prose is never
used for this statusless classification or retained in diagnostics.

### Author adjudication and revision trace boundary

`packages/schemas` also owns the strict, versioned author-adjudication plan and
Expand Down
15 changes: 10 additions & 5 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Product vision and roadmap

**Status:** Living document<br>
**Last reviewed:** 2026-09-13<br>
**Last reviewed:** 2026-09-15<br>
**Current stage:** Workflow parity and release (v0.9.0)

This document describes product direction, not fixed delivery dates. **Now** is
Expand Down Expand Up @@ -177,7 +177,7 @@ applications.
| Previous | Integration hardening and outcome validation ([v0.6.0](https://github.com/akoita/draft-loop/releases/tag/v0.6.0)) | Released; validation failed | Preserve a reproducible integrated baseline without overstating application readiness | Failed representative result carried into v0.7; see [stage evidence](stage-evidence-v0.6.0.md) |
| Previous | Evidence-backed CV drafting (v0.7 program) | [Released alpha.5 checkpoint](stage-evidence-v0.7.0-alpha.5.md); implementation history carried forward; outcome not validated | Produce a complete factual, source-traceable application draft | v0.8 candidate evidence now covers the bounded drafting and review vertical |
| Previous | Usable CV MVP ([v0.8.0-alpha.1](https://github.com/akoita/draft-loop/releases/tag/v0.8.0-alpha.1)) | [Released alpha](stage-evidence-v0.8.0-alpha.1.md); 17/17 issues closed; representative outcome not recorded | Produce one complete, factual, reviewed, human-approved, ATS-readable CV | Representative outcome evidence remains without overstating DOCX visual coverage |
| Now | Workflow parity and release ([milestone v0.9.0](https://github.com/akoita/draft-loop/milestone/4)) | [Twenty-four observations across two consented cases are indeterminate](consented-pilot-v0.9.md); parity not validated | Demonstrate the complete application-grade workflow and publish evidence | #350 informed corrections #351/#352; later attempts repeatedly failed before a draft. #376 corrects per-generation cap accounting, #380 adds fixed result diagnostics, and #393/#395 add private category capture. #397 again failed non-retryably on its first author attempt; capture established terminal reason `api_error` but no cause or product-quality evidence. #398 owns classification before another live attempt. #75/#250 remain blocked |
| Now | Workflow parity and release ([milestone v0.9.0](https://github.com/akoita/draft-loop/milestone/4)) | [Twenty-four observations across two consented cases are indeterminate](consented-pilot-v0.9.md); parity not validated | Demonstrate the complete application-grade workflow and publish evidence | #350 informed corrections #351/#352; later attempts repeatedly failed before a draft. #376 corrects per-generation cap accounting, #380 adds fixed result diagnostics, and #393/#395 add private category capture. #397 established terminal reason `api_error`; #398 classifies a statusless structured occurrence as bounded and retryable without consuming provider prose. A later live observation still requires fresh authorization. #75/#250 remain blocked |
| Later | Retrieval and provider quality | Integrated lexical baseline; partial components | Improve evidence selection and dependable live runs | Vector/hybrid comparison, cancellation, and provider recovery in the packaged path |
| Later | Broader real-application pilot | Implemented harness; not outcome-validated | Test factuality, quality, and effort across more cases | Consented cases, calibrated measures, and recorded limitations |
| Later | Production-ready beta | Partial implementation; not production-validated | Distribute a safe, dependable desktop application | Signed installers, safe migrations, recovery, accessibility, and platform evidence |
Expand Down Expand Up @@ -800,9 +800,13 @@ failed non-retryably with `unknown`, no failure stage or reason, and no artifact
or critic call. Fixed diagnostics retained only category classifications; the
private local capture established terminal reason `api_error` and retained no
subtype. Official Agent SDK sources document this terminal category, while a
`success` subtype is only an inference here. Issue #398 owns the provider-free
classification change. Authorization is exhausted, parity remains
indeterminate, and #75/#250 remain blocked.
`success` subtype is only an inference here. Issue #398 implements the provider-free
classification change: a structured `api_error` without a finite numeric API
status is now retryable within the existing orchestration cap, while fixed
diagnostics and status-first handling retain the privacy and failure bounds.
Authorization is exhausted, parity remains indeterminate, and #75/#250 remain
blocked. A later live observation requires a fresh bounded issue and explicit
authorization.

**Exit criterion:** The representative comparison records no factual-invariant
violations or unsupported model-added facts, preserves required sections and
Expand Down Expand Up @@ -904,6 +908,7 @@ issues retain implementation chronology.

| Date | Decision | Product implication |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 2026-09-15 | Classified statusless structured Claude API errors under #398. | The documented `api_error` terminal reason and error-result `success` subtype now produce fixed diagnostics. Without a finite numeric status, the failure is transient and retryable only within existing orchestration caps; numeric statuses retain precedence, provider prose remains excluded, and no live attempt is authorized. |
| 2026-09-15 | Recorded #397 as an indeterminate twenty-fourth matched-backend observation. | Both authentication probes passed, but one Anthropic author attempt failed non-retryably with `unknown`. Private bounded capture established terminal reason `api_error` but no subtype or underlying cause; no artifact or critic review occurred. Authorization is exhausted, #398 owns classification, and #75/#250 remain blocked. |
| 2026-09-15 | Routed opt-in Claude category capture through the local run boundary under #395. | A programmatic diagnostic caller can pass the private capture parent to Anthropic user-session run adapters without enabling capture for default, API-key, OpenAI, CLI, renderer, or persisted workspace paths. This provider-free plumbing does not authorize a live attempt. |
| 2026-09-14 | Added opt-in local capture for unknown Claude categories under #393. | Explicit diagnostic sessions can preserve only bounded, category-shaped unknown result subtype and terminal-reason strings in a private caller-owned file. Capture is disabled by default, durable history remains content-free, provider behavior is unchanged, and no live attempt is authorized. |
Expand Down
80 changes: 79 additions & 1 deletion packages/providers/src/user-session.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ const policy = {
sensitiveDataAcknowledged: false,
} as const;
const claudeErrorDiagnosticCases = [
{ field: "subtype", value: "success", code: "claude_error_subtype_success" },
{
field: "subtype",
value: "error_max_turns",
Expand All @@ -78,6 +79,7 @@ const claudeErrorDiagnosticCases = [
code: "claude_error_subtype_error_max_structured_output_retries",
},
{ field: "terminal_reason", value: "completed", code: "claude_terminal_reason_completed" },
{ field: "terminal_reason", value: "api_error", code: "claude_terminal_reason_api_error" },
{ field: "terminal_reason", value: "max_turns", code: "claude_terminal_reason_max_turns" },
{
field: "terminal_reason",
Expand Down Expand Up @@ -574,12 +576,88 @@ describe("AnthropicClaudeUserSessionAdapter", () => {
}
});

it.each([0, 1] as const)(
"maps statusless api_error results on Claude process exit %i to a retryable transient error",
async (exitCode) => {
const captureParent = await mkdtemp(join(tmpdir(), "draft-loop-claude-api-error-test-"));
const resultMarker = `private-api-error-result-${exitCode}`;
const errorsMarker = `private-api-error-errors-${exitCode}`;
const sessionMarker = `private-api-error-session-${exitCode}`;
const stderrMarker = `private-api-error-stderr-${exitCode}`;
try {
const error = await captureClaudeStructuredError(
{
subtype: "success",
terminal_reason: "api_error",
result: resultMarker,
errors: [errorsMarker],
session_id: sessionMarker,
},
{ captureParent, exitCode, stderr: stderrMarker },
);

expect(error).toMatchObject({ code: "transient", retryable: true, status: null });
expect(error.message).toBe("The user-session provider encountered a transient error.");
expect(error.diagnostics).toEqual([
{ code: "claude_error_subtype_success", path: "subtype" },
{ code: "claude_terminal_reason_api_error", path: "terminal_reason" },
{ code: "claude_stop_reason_unavailable", path: "stop_reason" },
]);
expect(await readdir(captureParent)).toEqual([]);
expect(error.diagnostics).not.toContainEqual(
expect.objectContaining({
code: expect.stringMatching(/^local_claude_category_capture_/u),
}),
);
for (const marker of [resultMarker, errorsMarker, sessionMarker, stderrMarker]) {
expect(error.message).not.toContain(marker);
expect(JSON.stringify(error.metadata)).not.toContain(marker);
expect(JSON.stringify(error.diagnostics)).not.toContain(marker);
expect(JSON.stringify(error)).not.toContain(marker);
}
} finally {
await rm(captureParent, { recursive: true, force: true });
}
},
);

it.each([
[400, "unknown", false, "The user-session provider request failed."],
[401, "authentication", false, "The user-session provider is not authenticated."],
[403, "authentication", false, "The user-session provider is not authenticated."],
[429, "rate-limit", true, "The user-session provider rate limit was reached."],
[500, "transient", true, "The user-session provider encountered a transient error."],
] as const)(
"keeps numeric Claude status %i ahead of api_error terminal reason",
async (status, code, retryable, message) => {
const resultMarker = `private-numeric-api-error-result-${status}`;
const error = await captureClaudeStructuredError(
{
subtype: "success",
terminal_reason: "api_error",
api_error_status: status,
result: resultMarker,
},
{ stderr: `private-numeric-api-error-stderr-${status}` },
);

expect(error).toMatchObject({ code, retryable, status, metadata: { status } });
expect(error.message).toBe(message);
expect(error.diagnostics).toEqual([
{ code: "claude_error_subtype_success", path: "subtype" },
{ code: "claude_terminal_reason_api_error", path: "terminal_reason" },
{ code: "claude_stop_reason_unavailable", path: "stop_reason" },
]);
expect(JSON.stringify(error)).not.toContain(resultMarker);
expect(JSON.stringify(error)).not.toContain(`private-numeric-api-error-stderr-${status}`);
},
);

it.each(claudeErrorDiagnosticCases)(
"maps allowlisted Claude error $field value $value to a fixed diagnostic",
async ({ field, value, code }) => {
const error = await captureClaudeStructuredError({ [field]: value });

expect(error).toMatchObject({ code: "unknown", retryable: false, status: null });
expect(error.diagnostics).toHaveLength(3);
expect(error.diagnostics).toContainEqual({ code, path: field });
expect(error.diagnostics?.map(({ path }) => path)).toEqual([
Expand Down
15 changes: 11 additions & 4 deletions packages/providers/src/user-session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -483,6 +483,7 @@ interface ClaudeJsonResult {
}

const claudeErrorSubtypeDiagnosticCodes = new Map([
["success", "claude_error_subtype_success"],
["error_max_turns", "claude_error_subtype_error_max_turns"],
["error_during_execution", "claude_error_subtype_error_during_execution"],
["error_max_budget_usd", "claude_error_subtype_error_max_budget_usd"],
Expand All @@ -493,6 +494,7 @@ const claudeErrorSubtypeDiagnosticCodes = new Map([
]);

const claudeTerminalReasonDiagnosticCodes = new Map([
["api_error", "claude_terminal_reason_api_error"],
["completed", "claude_terminal_reason_completed"],
["max_turns", "claude_terminal_reason_max_turns"],
["tool_deferred", "claude_terminal_reason_tool_deferred"],
Expand All @@ -507,10 +509,7 @@ const claudeTerminalReasonDiagnosticCodes = new Map([
["model_error", "claude_terminal_reason_model_error"],
]);

const claudeCategoryCaptureKnownSubtypes = new Set([
...claudeErrorSubtypeDiagnosticCodes.keys(),
"success",
]);
const claudeCategoryCaptureKnownSubtypes = new Set(claudeErrorSubtypeDiagnosticCodes.keys());
const claudeCategoryCaptureKnownTerminalReasons = new Set(
claudeTerminalReasonDiagnosticCodes.keys(),
);
Expand Down Expand Up @@ -601,6 +600,14 @@ function mapClaudeStructuredError(response: ClaudeJsonResult): ProviderAdapterEr
{ ...statusMetadata, retryable: true, diagnostics },
);
}
if (status === undefined && response.terminal_reason === "api_error") {
return new ProviderAdapterError(
"anthropic",
"transient",
"The user-session provider encountered a transient error.",
{ retryable: true, diagnostics },
);
}
return new ProviderAdapterError(
"anthropic",
"unknown",
Expand Down