Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -805,9 +805,10 @@ retry reset, candidate approval, or export is triggered by capture or replay.

### Local unknown Claude category capture

The Anthropic user-session adapter accepts an optional
`localClaudeCategoryCaptureParent` for explicit diagnostic sessions. Capture is
disabled by default, has no CLI or renderer control, and requires a
The local application driver accepts an optional
`localClaudeCategoryCaptureParent` for explicit diagnostic sessions and passes
it only to Anthropic user-session adapters constructed for run execution.
Capture is disabled by default, has no CLI or renderer control, and requires a
caller-selected parent directory that already exists. When a structured Claude
error contains an unrecognized category-shaped `subtype` or `terminal_reason`,
the adapter writes only those exact strings to a new `categories.json` file in
Expand Down
8 changes: 8 additions & 0 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -764,6 +764,13 @@ private material remain excluded; durable diagnostics record only fixed
capture-saved or capture-failed codes. This provider-free diagnostic does not
reinterpret #384 or authorize another live attempt.

Issue #395 routes that explicit capture parent through the local application
run boundary only when a diagnostic caller supplies it. Default workflows,
API-key adapters, OpenAI user sessions, CLI and renderer controls, persisted
workspace configuration, and provider behavior remain unchanged. This plumbing
makes #393 usable by a later bounded diagnostic workflow but neither performs
nor authorizes provider transmission.

The twenty-second bounded observation under #382 used revision
`349388de4820eca30543492d8ad1266199cdda3e` after explicit provider-transmission
authorization. Both 20-second authentication probes passed, but three
Expand Down Expand Up @@ -886,6 +893,7 @@ issues retain implementation chronology.

| Date | Decision | Product implication |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 2026-09-15 | Routed opt-in Claude category capture through the local run boundary under #395. | A programmatic diagnostic caller can pass the private capture parent to Anthropic user-session run adapters without enabling capture for default, API-key, OpenAI, CLI, renderer, or persisted workspace paths. This provider-free plumbing does not authorize a live attempt. |
| 2026-09-14 | Added opt-in local capture for unknown Claude categories under #393. | Explicit diagnostic sessions can preserve only bounded, category-shaped unknown result subtype and terminal-reason strings in a private caller-owned file. Capture is disabled by default, durable history remains content-free, provider behavior is unchanged, and no live attempt is authorized. |
| 2026-09-13 | Recorded #384 as an indeterminate twenty-third matched-backend observation. | After explicit authorization, both 20-second authentication probes passed, but one Anthropic author attempt failed non-retryably with `unknown` after 351,508 ms. Fixed diagnostics classify only result subtype, terminal reason, and stop reason; no artifact or review occurred, and authorization is exhausted. This adds no product-quality evidence; #75/#250 remain blocked. |
| 2026-09-13 | Recorded #382 as an indeterminate twenty-second matched-backend observation. | After explicit authorization, both 20-second auth probes passed, but three Anthropic author attempts failed with generic `invalid-response`; the first two were retryable and the final attempt was non-retryable with factual-invariant rejection. No artifact or review occurred; authorization is exhausted and #75/#250 remain blocked. |
Expand Down
279 changes: 279 additions & 0 deletions packages/application/src/local-claude-category-capture.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,279 @@
import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";

import type {
AnthropicClient,
OpenAIClient,
UserSessionProcessRunner,
} from "@draft-loop/providers";
import { openSqliteStorage } from "@draft-loop/storage";
import { describe, expect, it, vi } from "vitest";

import { createLocalApplicationDriver } from "./local.js";

const silent = { write: () => undefined };

async function providerWorkspace(prefix: string): Promise<string> {
const root = await mkdtemp(join(tmpdir(), prefix));
await mkdir(join(root, "evidence"));
await writeFile(join(root, "job.md"), "Build reliable TypeScript tools.\n", "utf8");
await writeFile(join(root, "evidence", "resume.md"), "Built TypeScript tools.\n", "utf8");
return root;
}

async function initializeWorkspace(
root: string,
authorCompany: "anthropic" | "openai",
criticCompany: "anthropic" | "openai",
): Promise<void> {
await createLocalApplicationDriver().initialize(
{
root,
jobDescription: "job.md",
sources: "evidence",
authorCompany,
authorModel: authorCompany === "anthropic" ? "claude-sonnet-4-5" : "gpt-5.6-luna",
criticCompany,
criticModel: criticCompany === "anthropic" ? "claude-sonnet-4-5" : "gpt-5.6-luna",
},
silent,
);
}

function claudeUnknownCategoryRunner(
subtype: string,
terminalReason: string,
): UserSessionProcessRunner {
return vi.fn(async () => ({
exitCode: 1,
stdout: JSON.stringify({
type: "result",
is_error: true,
subtype,
terminal_reason: terminalReason,
stop_reason: "stop_sequence",
api_error_status: 503,
result: "synthetic-private-result-marker",
errors: ["synthetic-private-errors-marker"],
session_id: "synthetic-private-session-marker",
usage: { private: "synthetic-private-usage-marker" },
structured_output: { private: "synthetic-private-structured-output-marker" },
prompt: "synthetic-private-prompt-marker",
path: "/synthetic/private/path-marker",
api_key: "synthetic-secret-credential-marker",
}),
stderr: "synthetic-private-stderr-marker",
}));
}

describe("local Claude category capture routing", () => {
it("routes only unknown Anthropic user-session categories to the configured local parent", async () => {
const root = await providerWorkspace("draft-loop-app-claude-capture-");
const captureParent = await mkdtemp(join(tmpdir(), "draft-loop-app-claude-capture-parent-"));
const subtype = "future_application_error_category";
const terminalReason = "future_application_terminal_reason";
const runner = claudeUnknownCategoryRunner(subtype, terminalReason);
const output: string[] = [];
const io = { write: (line: string) => output.push(line) };
const driver = createLocalApplicationDriver({
localClaudeCategoryCaptureParent: captureParent,
providerAuthModeConfiguration: { anthropic: "user-session", openai: "api-key" },
userSessionRunners: { anthropic: runner },
resolveCredential: async () => "synthetic-secret-credential-marker",
});

try {
await initializeWorkspace(root, "anthropic", "openai");
const snapshot = await driver.start({ root, allowProviderData: true }, io);

expect(snapshot).toMatchObject({
state: "provider-error",
lastError: {
code: "transient",
message: "The provider request failed. You can retry safely.",
provider: "anthropic",
step: "author",
retryable: true,
},
});
expect(snapshot.lastError?.diagnostics).toContainEqual({
code: "local_claude_category_capture_saved",
path: "local_claude_category_capture",
});
expect(runner).toHaveBeenCalledOnce();

const directories = await readdir(captureParent);
expect(directories).toHaveLength(1);
const directoryName = directories[0];
if (directoryName === undefined) throw new Error("Expected a category capture directory.");
const captureDirectory = join(captureParent, directoryName);
const captureText = await readFile(join(captureDirectory, "categories.json"), "utf8");
expect(JSON.parse(captureText)).toEqual({ subtype, terminal_reason: terminalReason });

const storage = openSqliteStorage(join(root, ".draft-loop", "history.sqlite"));
try {
const history = {
run: await storage.getRun(snapshot.runId),
events: await storage.listAuditEvents(snapshot.workspaceId),
};
const durableText = JSON.stringify({ snapshot, history, output });
for (const marker of [
subtype,
terminalReason,
captureParent,
directoryName,
"synthetic-private-result-marker",
"synthetic-private-errors-marker",
"synthetic-private-session-marker",
"synthetic-private-usage-marker",
"synthetic-private-structured-output-marker",
"synthetic-private-prompt-marker",
"/synthetic/private/path-marker",
"synthetic-secret-credential-marker",
"synthetic-private-stderr-marker",
]) {
expect(durableText).not.toContain(marker);
}
} finally {
await storage.close();
}
} finally {
await rm(root, { recursive: true, force: true });
await rm(captureParent, { recursive: true, force: true });
}
});

it("leaves capture disabled by default for an Anthropic user session", async () => {
const root = await providerWorkspace("draft-loop-app-claude-capture-default-");
const subtype = "future_default_category_marker";
const runner = claudeUnknownCategoryRunner(subtype, "future_default_terminal");
const output: string[] = [];

try {
await initializeWorkspace(root, "anthropic", "openai");
const snapshot = await createLocalApplicationDriver({
providerAuthModeConfiguration: { anthropic: "user-session", openai: "api-key" },
userSessionRunners: { anthropic: runner },
}).start({ root, allowProviderData: true }, { write: (line) => output.push(line) });

expect(snapshot).toMatchObject({
state: "provider-error",
lastError: {
code: "transient",
step: "author",
retryable: true,
},
});
expect(snapshot.lastError?.diagnostics).toContainEqual({
code: "claude_error_subtype_unrecognized",
path: "subtype",
});
expect(snapshot.lastError?.diagnostics).not.toContainEqual(
expect.objectContaining({
code: expect.stringMatching(/^local_claude_category_capture_/u),
}),
);
expect(runner).toHaveBeenCalledOnce();
expect(JSON.stringify({ snapshot, output })).not.toContain(subtype);
expect(JSON.stringify({ snapshot, output })).not.toContain("future_default_terminal");
} finally {
await rm(root, { recursive: true, force: true });
}
});

it("does not create capture output for Anthropic API-key authentication", async () => {
const root = await providerWorkspace("draft-loop-app-anthropic-api-key-capture-");
const captureParent = await mkdtemp(join(tmpdir(), "draft-loop-app-anthropic-api-key-parent-"));
const output: string[] = [];
const create = vi.fn(async () => {
throw new Error("synthetic Anthropic API-key failure");
});
const credentials = vi.fn(async () => "synthetic-anthropic-api-key");
const driver = createLocalApplicationDriver({
localClaudeCategoryCaptureParent: captureParent,
providerAuthModeConfiguration: { anthropic: "api-key", openai: "api-key" },
resolveCredential: credentials,
providerClientFactories: {
anthropic: () => ({ messages: { create } }) as unknown as AnthropicClient,
},
});

try {
await initializeWorkspace(root, "anthropic", "openai");
const snapshot = await driver.start(
{ root, allowProviderData: true },
{ write: (line) => output.push(line) },
);

expect(snapshot).toMatchObject({
state: "provider-error",
lastError: { provider: "anthropic", step: "author" },
});
expect(credentials).toHaveBeenCalledOnce();
expect(credentials).toHaveBeenCalledWith("anthropic");
expect(create).toHaveBeenCalledOnce();
expect(await readdir(captureParent)).toEqual([]);
expect(JSON.stringify({ snapshot, output })).not.toContain(captureParent);
} finally {
await rm(root, { recursive: true, force: true });
await rm(captureParent, { recursive: true, force: true });
}
});

it.each(["user-session", "api-key"] as const)(
"does not route capture through the OpenAI %s adapter",
async (authMode) => {
const root = await providerWorkspace(`draft-loop-app-openai-capture-${authMode}-`);
const captureParent = await mkdtemp(join(tmpdir(), "draft-loop-app-openai-capture-parent-"));
const output: string[] = [];
const driver = createLocalApplicationDriver({
localClaudeCategoryCaptureParent: captureParent,
providerAuthModeConfiguration: { anthropic: "api-key", openai: authMode },
resolveCredential: async () => "synthetic-openai-api-key",
...(authMode === "user-session"
? {
userSessionRunners: {
openai: vi.fn<UserSessionProcessRunner>(async () => ({
exitCode: 1,
stdout: "synthetic OpenAI user-session failure",
stderr: "",
})),
},
}
: {
providerClientFactories: {
openai: () =>
({
responses: {
create: async () => {
throw new Error("synthetic OpenAI API-key failure");
},
},
}) as OpenAIClient,
},
}),
});

try {
await initializeWorkspace(root, "openai", "anthropic");
const snapshot = await driver.start(
{ root, allowProviderData: true },
{
write: (line) => output.push(line),
},
);

expect(snapshot).toMatchObject({
state: "provider-error",
lastError: { provider: "openai", step: "author" },
});
expect(await readdir(captureParent)).toEqual([]);
expect(JSON.stringify({ snapshot, output })).not.toContain(captureParent);
} finally {
await rm(root, { recursive: true, force: true });
await rm(captureParent, { recursive: true, force: true });
}
},
);
});
Loading