diff --git a/docs/plans/nested-instruction-discovery.md b/docs/plans/nested-instruction-discovery.md new file mode 100644 index 0000000000..0ef30cdd7e --- /dev/null +++ b/docs/plans/nested-instruction-discovery.md @@ -0,0 +1,9 @@ +# Referenced-path instruction discovery + +PR #1976 originally performed a serial full-tree scan at the start of every turn, even when a prompt referenced one file. On a macOS Node 24.20 fixture with 100 groups of 100 directories, five fresh-turn activations took 650, 614, 773, 596 and 633 ms; same-turn cache hits took less than 1 ms. + +Turn-time discovery now reads only each referenced path's ancestor scopes, with at most 64 distinct context paths per call and 16 ancestor levels. Concurrent tool calls share in-flight reads of each scope. Missing instruction files are cached too. Every turn owns a fresh cache; a file-tool write, rename or deletion of AGENTS.md invalidates that execution root. A previously unseen scope is read on first entry. Already-read scopes changed through an external editor or shell are refreshed next turn. Already-injected rules remain fixed for the current turn; invalidating discovery permits new activations, not replacement of existing prompt instructions. No filesystem watcher or cross-turn cache is trusted for prompt correctness. + +Settings retains the bounded full inventory and its existing 30-second cache. Instruction precedence, workspace trust, directory-symlink exclusion, nested checkout boundaries, generated-directory exclusion, file-size limits and prompt budgets remain enforced. A symlinked instruction file still passes through the existing canonical-path read boundary. The existing handwritten filter predicate is replaced with TypeScript's inferred predicate. + +The same fixture after the change measured fresh-turn activations of 14, 13, 5, 8 and 6 ms while other validation ran. These are local measurements, not a cross-platform latency guarantee. Run `pnpm test -- nested-instruction-latency` to repeat the real public activation-path benchmark; it prints each fresh-turn and cached duration. `pnpm test -- project-instructions` covers scope freshness, explicit invalidation, trust and repository/symlink boundaries. diff --git a/docs/ui-taste.md b/docs/ui-taste.md index b7f7610a7b..c075827a13 100644 --- a/docs/ui-taste.md +++ b/docs/ui-taste.md @@ -767,6 +767,15 @@ left-elision (`direction: rtl` + `text-overflow: ellipsis`, same trick as `.git- the leaf stays visible; mirror the full path on the row's `title` for the native tooltip. Spec: [`tests/e2e/settings-sources-skills.e2e.ts`](../tests/e2e/settings-sources-skills.e2e.ts). +Nested instruction rows use the badge as activation state, not origin: **active** uses the existing +accent outline when the latest turn selected that scope; **scoped** uses the quiet default outline when +the file is available but unrelated to that turn; **duplicate** (quiet outline) marks a nested file +whose text repeats one already listed, so it is loaded once through that one. Keep the governed +directory and the explanatory state in the detail line so sibling scopes are understandable without +adding another row of chips. When discovery stopped at its cap, a single `.sources-empty` line under +the list says the list may be incomplete — a note, not a row. +Spec: [`tests/e2e/settings-sources-nested-instructions.e2e.ts`](../tests/e2e/settings-sources-nested-instructions.e2e.ts). + ## Prove visual changes with a focused e2e eval Per `AGENTS.md`, any user-visible change needs a focused WebdriverIO Electron spec that seeds the diff --git a/docs/user/README.md b/docs/user/README.md index b2d0c6f549..188c26f5e1 100644 --- a/docs/user/README.md +++ b/docs/user/README.md @@ -13,6 +13,7 @@ contributor and design archive — start here instead. - [Install](install.md) — macOS 26+ from a signed DMG, or run from source - [Quickstart](quickstart.md) — open a project, pick a model, send the first prompt - [Connect a model](connect-a-model.md) — API key, environment scan, or a local server +- [Project instructions](project-instructions.md) — root and directory-scoped AGENTS.md files ## Staying in control diff --git a/docs/user/project-instructions.md b/docs/user/project-instructions.md new file mode 100644 index 0000000000..5b8a4b58ef --- /dev/null +++ b/docs/user/project-instructions.md @@ -0,0 +1,57 @@ +--- +title: Project instructions +description: Give Copse global, project-wide, or directory-scoped guidance with AGENTS.md files. +--- + +# Project instructions + +Instruction files let a repository carry its own build commands, conventions, and safety notes. +Copse lists every discovered source under **Settings → Customise → Instruction files**. Project +instructions stay inert until you trust the project; click a file name there to read it first. + +## Project-wide instructions + +At the project root, Copse reads these files in order: + +1. `AGENT.md` +2. `AGENTS.md` +3. `CLAUDE.md` + +Identical contents are injected once. Global `~/AGENTS.md` and `~/.claude/CLAUDE.md` load beneath +the project layer as user-owned guidance. + +## Directory-scoped instructions + +A nested `AGENTS.md` applies only when a path under its directory enters the turn's context. A +path enters context when the prompt or an attachment names it, or when one of Copse's built-in +file tools touches it: `read_file`, `list_dir`, `search_code`, `search_codebase`, +`read_staged_diff`, `write_file`, `str_replace`, `delete_file`, `rename_file`, and +`make_directory`. Nothing else activates a nested file: a `run_shell` command that reads or writes +under the directory, an ACP agent's own file access, and a subagent's reads do not count. +Instructions are applied from the project root toward the target directory, so the nearest file +appears last and can refine broader conventions. Sibling scopes stay inactive unless that sibling +has a relevant path. + +When an edit tool is the first action to enter a new scope, Copse loads the applicable instruction +chain and defers that edit once. The agent sees the new rules and retries instead of changing the +file before its local guidance is available. Each activation adds a one-line note to the +transcript naming the file that was loaded. + +During a turn, Copse reads only the ancestor directories of referenced paths, rather than scanning +unrelated parts of the repository. Each scope, including a missing `AGENTS.md`, is read once when +first referenced and shared by later tool calls. Writing, moving, or removing an `AGENTS.md` with a +file tool invalidates those reads, so a later tool can activate newly available instructions. Rules +already injected remain in the current turn; changes to those rules apply next turn. External +edits to an already-read scope are also seen next turn. A newly referenced scope is read when the +agent first enters it. Settings still discovers the full bounded inventory. + +Nested `AGENT.md` and `CLAUDE.md` remain root-only compatibility formats. Only `AGENTS.md` follows +the cross-client directory-scoping convention, which avoids silently changing the meaning of +vendor-specific files. + +Sources marks a nested file **active** when the latest turn used it, **scoped** when it is +available but did not apply, and **duplicate** when its text repeats a file already listed (the +text is loaded once, through that file). Discovery skips dependency, generated, vendored, cache, +nested-repo, and VCS trees; it does not follow a symlink outside the trusted workspace. Very deep +or unusually large instruction trees are bounded so they cannot consume the whole prompt; when +discovery stops at that bound, Sources says the list may be incomplete. diff --git a/schemas/api-protocol.manifest.json b/schemas/api-protocol.manifest.json index ad822a3160..508e610d75 100644 --- a/schemas/api-protocol.manifest.json +++ b/schemas/api-protocol.manifest.json @@ -1,6 +1,6 @@ { "$comment": "Generated by scripts/gen-api-protocol.mts from ApiClient (src/preload/api.d.ts) and the preload bindings; do not edit by hand. The full JSON Schema is a build output (dist/schemas/api-protocol.schema.json). See docs/api-protocol.md.", - "version": 4, + "version": 5, "channels": { "invoke": { "acp:auto-setup": { diff --git a/src/main/ipc/register-handlers.ts b/src/main/ipc/register-handlers.ts index 2c708e84d9..4d6e626897 100644 --- a/src/main/ipc/register-handlers.ts +++ b/src/main/ipc/register-handlers.ts @@ -2142,13 +2142,34 @@ export function registerAllHandlers(win: BrowserWindow, registry: ToolRegistry): return listUnsandboxedProjectHooks(root) }) ipcMain.handle('instructions:list', async () => - (await loadProjectInstructionSources()).map(({ path, name, scope, content, active }) => ({ - path, - name, - scope, - bytes: Buffer.byteLength(content, 'utf-8'), - active, - })), + ( + await loadProjectInstructionSources({ + useLatestNestedActivation: true, + refreshNestedDiscovery: true, + }) + ).map( + ({ + path, + name, + scope, + content, + active, + trusted, + scopePath, + duplicateOf, + discoveryTruncated, + }) => ({ + path, + name, + scope, + bytes: Buffer.byteLength(content, 'utf-8'), + active, + trusted, + ...(scopePath !== undefined ? { scopePath } : {}), + ...(duplicateOf !== undefined ? { duplicateOf } : {}), + ...(discoveryTruncated ? { discoveryTruncated } : {}), + }), + ), ) /** * Read one instruction file for display (Settings → Sources opens it in the diff --git a/src/main/services/agent-service.test.ts b/src/main/services/agent-service.test.ts index 14c85a3070..a34fda95e7 100644 --- a/src/main/services/agent-service.test.ts +++ b/src/main/services/agent-service.test.ts @@ -1,5 +1,9 @@ import { describe, it } from 'node:test' import assert from 'node:assert/strict' +import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { z } from 'zod' import * as agentService from './agent-service.ts' import * as providerSelection from './providers/provider-selection.ts' import { suggestThreadTitle } from './title-generator.ts' @@ -17,6 +21,8 @@ import { type PluginToolRuntimeController, } from './plugins/plugin-tool-controller.ts' import { pluginModelValue } from '@shared/plugin-model.ts' +import { defineTool } from '@shared/types' +import { runWithWorkspaceTrust } from './security/workspace-trust.ts' // agent-service is now an orchestrator that re-exports the public surface from the // focused modules it composes. These tests pin that public surface so IPC callers @@ -242,6 +248,246 @@ describe('runAgent AgentHost decoupling', () => { assert.deepEqual(checkpoints.at(-1), result.messages.slice(0, checkpoints.at(-1)?.length)) }) + it('activates nested instructions on first file access and defers the first edit', async () => { + const root = await mkdtemp(join(tmpdir(), 'copse-agent-nested-instructions-')) + await mkdir(join(root, 'packages', 'api'), { recursive: true }) + await writeFile(join(root, 'packages', 'api', 'AGENTS.md'), 'Never edit before reading this.') + const writes: string[] = [] + const registry = new ToolRegistry() + registry.register( + defineTool({ + name: 'write_file', + description: 'Test edit tool', + parameters: z.object({ path: z.string() }), + execute: ({ path }) => { + writes.push(path) + return Promise.resolve('File written.') + }, + }), + ) + + let calls = 0 + const provider: LLMProvider = { + stream: async function* (messages) { + calls += 1 + const system = messages.find((message) => message.role === 'system') + assert.ok(system?.role === 'system') + if (calls === 1) { + assert.doesNotMatch(system.content, /Never edit before reading this/) + yield { + type: 'tool_call' as const, + toolCall: { + id: 'first-edit', + name: 'write_file', + args: { path: 'packages/api/router.ts' }, + }, + } + return + } + assert.match(system.content, /Never edit before reading this/) + if (calls === 2) { + const deferred = messages.find( + (message) => + message.role === 'tool' && + message.toolResults.some((result) => result.toolCallId === 'first-edit'), + ) + assert.ok(deferred?.role === 'tool') + assert.match(deferred.toolResults[0]?.result ?? '', /Edit deferred/) + assert.deepEqual(writes, []) + yield { + type: 'tool_call' as const, + toolCall: { + id: 'retried-edit', + name: 'write_file', + args: { path: 'packages/api/router.ts' }, + }, + } + return + } + assert.deepEqual(writes, ['packages/api/router.ts']) + yield { type: 'text' as const, text: 'Done.' } + }, + } + setDefaultPluginRegistry(new PluginRegistry()) + await setSetting('subagentsEnabled', false) + await setSetting('skillsEnabled', false) + + try { + await runWithWorkspaceTrust(root, true, () => + runWithThreadExecutionContext( + { + projectId: 'project-nested', + threadId: 'thread-nested', + projectRoot: root, + root, + checkoutMode: 'shared', + branch: null, + }, + () => + runWithActiveRunIdentity('thread-nested', () => + agentService.runAgent( + 'thread-nested', + 'Make the requested change.', + [], + { emit: () => undefined }, + registry, + { + provider, + contextWindow: 100_000, + model: 'claude-sonnet-4-6', + maxSteps: 6, + maxLlmCalls: 6, + }, + ), + ), + ), + ) + assert.equal(calls, 3) + assert.deepEqual(writes, ['packages/api/router.ts']) + } finally { + setDefaultPluginRegistry(null) + await rm(root, { recursive: true, force: true }) + } + }) + + it('memoizes referenced instruction scopes, notices activations, and refreshes after an AGENTS.md write', async () => { + const root = await mkdtemp(join(tmpdir(), 'copse-agent-nested-discovery-')) + await mkdir(join(root, 'packages', 'api'), { recursive: true }) + await mkdir(join(root, 'packages', 'web'), { recursive: true }) + await writeFile(join(root, 'packages', 'api', 'AGENTS.md'), 'API rules here.') + const registry = new ToolRegistry() + registry.register( + defineTool({ + name: 'read_file', + description: 'Test read tool', + parameters: z.object({ path: z.string() }), + execute: () => Promise.resolve('contents'), + }), + ) + registry.register( + defineTool({ + name: 'write_file', + description: 'Test write tool', + parameters: z.object({ path: z.string(), content: z.string() }), + execute: async ({ path, content }) => { + await writeFile(join(root, path), content) + return 'File written.' + }, + }), + ) + + const readTool = ( + id: string, + path: string, + ): { type: 'tool_call'; toolCall: { id: string; name: string; args: { path: string } } } => ({ + type: 'tool_call', + toolCall: { id, name: 'read_file', args: { path } }, + }) + let calls = 0 + const provider: LLMProvider = { + stream: async function* (messages) { + calls += 1 + const system = messages.find((message) => message.role === 'system') + assert.ok(system?.role === 'system') + switch (calls) { + case 1: + assert.doesNotMatch(system.content, /API rules here/) + yield readTool('read-api', 'packages/api/a.ts') + return + case 2: + assert.match(system.content, /API rules here/) + // The prompt already referenced this missing scope. An external + // write stays cached until an explicit file-tool invalidation. + await writeFile(join(root, 'packages', 'web', 'AGENTS.md'), 'Web rules here.') + yield readTool('read-web-stale', 'packages/web/b.ts') + return + case 3: + assert.doesNotMatch(system.content, /Web rules here/) + yield { + type: 'tool_call' as const, + toolCall: { + id: 'write-web-agents', + name: 'write_file', + args: { path: 'packages/web/AGENTS.md', content: 'Web rules here.' }, + }, + } + return + case 4: + assert.doesNotMatch(system.content, /Web rules here/) + // A different path than the stale read: the loop skips a repeat of + // a recent call's exact arguments. + yield readTool('read-web-fresh', 'packages/web/c.ts') + return + default: + assert.match(system.content, /Web rules here/) + yield { type: 'text' as const, text: 'Done.' } + } + }, + } + const received: StreamChunk[] = [] + setDefaultPluginRegistry(new PluginRegistry()) + await setSetting('subagentsEnabled', false) + await setSetting('skillsEnabled', false) + + try { + await runWithWorkspaceTrust(root, true, () => + runWithThreadExecutionContext( + { + projectId: 'project-nested-discovery', + threadId: 'thread-nested-discovery', + projectRoot: root, + root, + checkoutMode: 'shared', + branch: null, + }, + () => + runWithActiveRunIdentity('thread-nested-discovery', () => + agentService.runAgent( + 'thread-nested-discovery', + 'Look around packages/web/b.ts.', + [], + { emit: (_threadId, chunk) => received.push(chunk) }, + registry, + { + provider, + contextWindow: 100_000, + model: 'claude-sonnet-4-6', + maxSteps: 8, + maxLlmCalls: 8, + }, + ), + ), + ), + ) + assert.equal(calls, 5) + + // One transcript line per activation, landing after that call's result — + // never between a tool call and its result, where it would strand the card. + const notices = received.flatMap((chunk, index) => + chunk.type === 'text' && chunk.text.includes('Loaded directory-scoped instructions') + ? [{ index, text: chunk.text }] + : [], + ) + assert.deepEqual( + notices.map((notice) => notice.text), + [ + '_Loaded directory-scoped instructions from `packages/api/AGENTS.md`._\n\n', + '_Loaded directory-scoped instructions from `packages/web/AGENTS.md`._\n\n', + ], + ) + const resultIndex = (toolCallId: string): number => + received.findIndex( + (chunk) => chunk.type === 'tool_result' && chunk.toolCallId === toolCallId, + ) + assert.ok(resultIndex('read-api') >= 0) + assert.ok(notices[0] && notices[0].index > resultIndex('read-api')) + assert.ok(notices[1] && notices[1].index > resultIndex('read-web-fresh')) + } finally { + setDefaultPluginRegistry(null) + await rm(root, { recursive: true, force: true }) + } + }) + it('emits a structured terminal record with raw provider failure details', async () => { const received: StreamChunk[] = [] const host: AgentHost = { diff --git a/src/main/services/agent-service.ts b/src/main/services/agent-service.ts index 15b5bafaef..5cd02a1bad 100644 --- a/src/main/services/agent-service.ts +++ b/src/main/services/agent-service.ts @@ -43,7 +43,12 @@ import { } from './agent-errors.ts' import { normalizeStopReason } from '@copse/agent/headless-contract.ts' import { resolveParentGoal } from '@copse/agent/working-brief.ts' -import { buildSystemPrompt } from './agent-system-prompt.ts' +import { buildSystemPromptWithMetadata } from './agent-system-prompt.ts' +import { + activateNestedInstructionSources, + createNestedInstructionTurn, + invalidateNestedInstructionDiscoveryForWrite, +} from './project-instructions.ts' import { hasLastUsage } from '@copse/llm/provider-usage.ts' import { clearActiveRunThread, @@ -393,6 +398,42 @@ function promptTextForSubmit(userPrompt: UserContent): string { .join('\n') } +/** Built-in file tools whose path establishes directory-scoped instruction context. */ +const INSTRUCTION_CONTEXT_PATH_FIELDS: Readonly> = { + read_file: ['path'], + list_dir: ['path'], + search_code: ['path'], + search_codebase: ['path'], + read_staged_diff: ['path'], + write_file: ['path'], + str_replace: ['path'], + delete_file: ['path'], + rename_file: ['from', 'to'], + make_directory: ['path'], +} + +function instructionContextPathsForTool(name: string, args: unknown): string[] { + const fields = INSTRUCTION_CONTEXT_PATH_FIELDS[name] + if (!fields || !isRecord(args)) return [] + return fields.flatMap((field) => { + const value = args[field] + return typeof value === 'string' && value.trim() ? [value] : [] + }) +} + +/** + * One transcript line per nested AGENTS.md that joined the prompt mid-turn, in + * the same italic-note voice as the model-fallback notice. Emitted as a text + * chunk between steps — never between a `tool_call` and its `tool_result`, + * where a text chunk would start a new assistant bubble and strand the card. + */ +function nestedInstructionsActivatedNotice(names: readonly string[]): string { + return ( + names.map((name) => `_Loaded directory-scoped instructions from \`${name}\`._`).join('\n') + + '\n\n' + ) +} + /** * Fire `beforeSubmitPrompt` (B1). Returns the user-facing notice to show when a * hook halted the submit (`continue: false`), or null to proceed. Recording is @@ -1513,13 +1554,36 @@ export async function runAgent( ? options.invokedAgent : undefined - const systemPrompt = await buildSystemPrompt({ + // One nested-AGENTS.md walk per turn: the prompt build below seeds the memo + // and every file tool call of this turn reuses it. + const nestedInstructionTurn = createNestedInstructionTurn() + const systemPromptBuild = await buildSystemPromptWithMetadata({ subagentsEnabled, invokedSkills, threadId, userPrompt: outboundPrompt, model, + trackInstructionActivation: true, + nestedInstructionTurn, }) + const systemPrompt = systemPromptBuild.prompt + const activeNestedInstructionPaths = new Set( + systemPromptBuild.instructionMetadata.activeNestedPaths, + ) + const activeInstructionContents = new Set( + systemPromptBuild.instructionMetadata.activeInstructionContents, + ) + let activeNestedInstructionBytes = systemPromptBuild.instructionMetadata.activeNestedBytes + // Names of nested files activated by a tool call, held until the next step + // boundary so the notice lands after that call's tool_result. + const pendingNestedInstructionNotices: string[] = [] + const flushNestedInstructionNotices = (): void => { + if (pendingNestedInstructionNotices.length === 0) return + sendChunk({ + type: 'text', + text: nestedInstructionsActivatedNotice(pendingNestedInstructionNotices.splice(0)), + }) + } const messages: LLMMessage[] = [ { role: 'system', content: systemPrompt }, @@ -1828,6 +1892,39 @@ export async function runAgent( signal: AbortSignal, toolCallId: string, ): Promise => { + const instructionContextPaths = instructionContextPathsForTool(name, args) + if (instructionContextPaths.length > 0) { + const activation = await activateNestedInstructionSources( + instructionContextPaths, + activeNestedInstructionPaths, + activeInstructionContents, + activeNestedInstructionBytes, + nestedInstructionTurn, + ) + for (const path of activation.activatedPaths) { + activeNestedInstructionPaths.add(path) + } + for (const content of activation.injectedContents) { + activeInstructionContents.add(content) + activeNestedInstructionBytes += Buffer.byteLength(content, 'utf-8') + } + pendingNestedInstructionNotices.push(...activation.injectedNames) + if (activation.block) { + const leadingSystem = trimmed.find((message) => message.role === 'system') + if (leadingSystem?.role === 'system') { + leadingSystem.content += `\n\n---\n\n${activation.block}` + } + // An edit must not land before the newly applicable rules have + // reached the model. Reads can proceed: their result and the new + // leading-system block arrive together before the next step. + if (isEditTool(name)) { + return ( + 'Edit deferred because this path activated nested AGENTS.md instructions. ' + + 'Review the newly loaded workspace instructions, then retry the same edit.' + ) + } + } + } if (isEditTool(name)) turnChangedFiles = true if (name === 'explore' && subagentsEnabled) { // ALS-scoped (not a global slot): the loop runs fanned-out @@ -1943,6 +2040,15 @@ export async function runAgent( const startedAt = Date.now() try { const raw = await runParentTool(name, args, signal, toolCallId) + // The agent just wrote, moved, or removed an AGENTS.md: the turn's + // discovery memo no longer describes the tree, so the next file tool + // call re-walks. `run_shell` writes are not seen here (documented). + if (isEditTool(name)) { + invalidateNestedInstructionDiscoveryForWrite( + instructionContextPathsForTool(name, args), + nestedInstructionTurn, + ) + } fireAfterToolUseHook({ threadId, turnTreeId, @@ -1999,6 +2105,9 @@ export async function runAgent( // `messages` above is `trimmed`, mutated in place as turns land, so // this persists everything the previous step produced. checkpointHistory() + // The previous step's tool results have been streamed; a notice + // here cannot come between a tool call and its result. + flushNestedInstructionNotices() }, recordStreamCut: (record) => { recordStreamCut(record, model) @@ -2028,6 +2137,9 @@ export async function runAgent( } }, }) + // A loop that stopped on a tool step (step cap, abort) still owes + // the notice for what that step activated. + flushNestedInstructionNotices() const subUsage = getAccumulatedSubagentUsage() if (subUsage.inputTokens || subUsage.outputTokens) { @@ -2085,6 +2197,7 @@ export async function runAgent( onLlmCall: (count: number): void => { setHookRunStep(count) checkpointHistory() + flushNestedInstructionNotices() }, recordStreamCut: (record) => { recordStreamCut(record, model) diff --git a/src/main/services/agent-system-prompt.test.ts b/src/main/services/agent-system-prompt.test.ts index f171143c5a..ffb07baaf7 100644 --- a/src/main/services/agent-system-prompt.test.ts +++ b/src/main/services/agent-system-prompt.test.ts @@ -148,6 +148,23 @@ describe('buildSystemPrompt instruction layers', () => { assert.match(prompt, /not trusted/) assert.ok(prompt.includes('Global user rules')) }) + + it('activates a nested AGENTS.md from a path in the current user turn only', async () => { + await mkdir(join(tempRoot, 'packages', 'api'), { recursive: true }) + await mkdir(join(tempRoot, 'packages', 'web'), { recursive: true }) + await writeFile(join(tempRoot, 'packages', 'api', 'AGENTS.md'), 'Use API conventions.') + await writeFile(join(tempRoot, 'packages', 'web', 'AGENTS.md'), 'Use web conventions.') + const prompt = await runWithWorkspaceTrust(tempRoot, true, () => + buildSystemPrompt({ + subagentsEnabled: false, + invokedSkills: [], + userPrompt: 'Please update packages/api/src/router.ts.', + }), + ) + assert.match(prompt, /path="packages\/api\/AGENTS\.md"/) + assert.match(prompt, /Use API conventions/) + assert.doesNotMatch(prompt, /Use web conventions/) + }) }) describe('buildSystemPrompt working directory', () => { diff --git a/src/main/services/agent-system-prompt.ts b/src/main/services/agent-system-prompt.ts index 30c5c484a1..90955aeae1 100644 --- a/src/main/services/agent-system-prompt.ts +++ b/src/main/services/agent-system-prompt.ts @@ -1,4 +1,9 @@ -import { loadAgentRequestedRulesCatalog, loadInstructionLayers } from './project-instructions.ts' +import { + loadAgentRequestedRulesCatalog, + loadInstructionLayersWithMetadata, + type InstructionLayerMetadata, + type NestedInstructionTurn, +} from './project-instructions.ts' import { getSetting, getSettingTrimmed } from './storage/settings.ts' import { getAgentExecutionRoot } from './execution-root.ts' import { getThreadExecutionContext } from './thread-execution-context.ts' @@ -75,8 +80,7 @@ async function buildRepositoryContext(): Promise { return `\nGit repository root: ${repositoryRoot}${subdirNote}` } -/** Assemble the system prompt for a run from base prompt + skills + instructions. */ -export async function buildSystemPrompt(opts: { +export interface BuildSystemPromptOptions { subagentsEnabled: boolean invokedSkills: string[] threadId?: string @@ -88,15 +92,40 @@ export async function buildSystemPrompt(opts: { * and the prompt stays model-agnostic. */ model?: string -}): Promise { + /** Persist nested-source activation for Settings; real local turns set this. */ + trackInstructionActivation?: boolean + /** + * The turn's nested-discovery memo. The build walks the tree once into it so + * the turn's tool calls reuse that walk; estimates omit it and share a cache. + */ + nestedInstructionTurn?: NestedInstructionTurn +} + +export interface SystemPromptBuildResult { + prompt: string + instructionMetadata: InstructionLayerMetadata +} + +/** Assemble the system prompt and retain nested-instruction runtime metadata. */ +export async function buildSystemPromptWithMetadata( + opts: BuildSystemPromptOptions, +): Promise { const { subagentsEnabled, invokedSkills, threadId } = opts const skillsToolsLine = buildSkillsToolsPromptLine() const userText = opts.userPrompt != null ? userContentToText(opts.userPrompt) : '' + const contextPaths = extractContextPathsFromText(userText) const cursorRuleContext: CursorRuleContext = { - contextPaths: extractContextPathsFromText(userText), + contextPaths, userText, } - const instructionLayers = await loadInstructionLayers({ cursorRuleContext }) + const instructionLayers = await loadInstructionLayersWithMetadata( + { + cursorRuleContext, + nestedContextPaths: contextPaths, + ...(opts.nestedInstructionTurn ? { nestedInstructionTurn: opts.nestedInstructionTurn } : {}), + }, + opts.trackInstructionActivation ?? false, + ) const agentRulesCatalog = await loadAgentRequestedRulesCatalog() const basePrompt = subagentsEnabled ? BASE_SYSTEM_PROMPT : BASE_SYSTEM_PROMPT_DIRECT_READS @@ -118,7 +147,7 @@ export async function buildSystemPrompt(opts: { (threadId ? hasTerminalSessions(threadId) : hasTerminalSessions()) const customInstructions = getSettingTrimmed('customInstructions') const opus5 = opts.model != null && isOpus5Model(opts.model) - return ( + const prompt = basePrompt .replace('{SKILLS_TOOLS_LINE}', skillsToolsLine) // Must be the agent execution root, not the renderer workspace root: the @@ -147,5 +176,10 @@ export async function buildSystemPrompt(opts: { (instructionLayers.global ? `\n\n---\n\n## User instructions\n\n${instructionLayers.global}` : '') - ) + return { prompt, instructionMetadata: instructionLayers.metadata } +} + +/** Prompt-only compatibility wrapper for estimates, tests, and other callers. */ +export async function buildSystemPrompt(opts: BuildSystemPromptOptions): Promise { + return (await buildSystemPromptWithMetadata(opts)).prompt } diff --git a/src/main/services/context-estimate.ts b/src/main/services/context-estimate.ts index 2e4077b0c7..87edea5c65 100644 --- a/src/main/services/context-estimate.ts +++ b/src/main/services/context-estimate.ts @@ -57,6 +57,7 @@ export async function estimateContextBreakdown( subagentsEnabled, invokedSkills: input.invokedSkills, model, + userPrompt: input.draftText, }) // Skill blocks are part of the system prompt string; measure them separately so // they can be attributed to "Skills" instead of inflating "System prompt". diff --git a/src/main/services/nested-instruction-latency.test.ts b/src/main/services/nested-instruction-latency.test.ts new file mode 100644 index 0000000000..5cfcf87012 --- /dev/null +++ b/src/main/services/nested-instruction-latency.test.ts @@ -0,0 +1,61 @@ +import { it } from 'node:test' +import assert from 'node:assert/strict' +import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises' +import { join } from 'node:path' +import { tmpdir } from 'node:os' +import { + activateNestedInstructionSources, + createNestedInstructionTurn, +} from './project-instructions.ts' +import { setWorkspaceRootForTest } from './workspace.ts' +import { runWithWorkspaceTrust } from './security/workspace-trust.ts' + +it('measures turn-start discovery against a 10,000-directory fixture', async () => { + const root = await mkdtemp(join(tmpdir(), 'copse-nested-latency-')) + const restore = setWorkspaceRootForTest(root) + try { + for (let group = 0; group < 100; group++) { + await Promise.all( + Array.from({ length: 100 }, (_, leaf) => + mkdir( + join( + root, + `group-${String(group).padStart(3, '0')}`, + `leaf-${String(leaf).padStart(3, '0')}`, + ), + { recursive: true }, + ), + ), + ) + } + await writeFile(join(root, 'group-000', 'AGENTS.md'), 'Use the scoped instructions.') + const rounds = [] + for (let round = 0; round < 5; round++) { + const turn = createNestedInstructionTurn() + const activate = (): ReturnType => + runWithWorkspaceTrust(root, true, () => + activateNestedInstructionSources( + ['group-000/leaf-000/example.ts'], + new Set(), + new Set(), + 0, + turn, + ), + ) + const start = performance.now() + const result = await activate() + const firstMs = performance.now() - start + assert.equal(result.injectedNames[0], 'group-000/AGENTS.md') + const again = performance.now() + await activate() + rounds.push({ firstMs, memoMs: performance.now() - again }) + } + console.log( + 'NESTED_INSTRUCTION_LATENCY', + JSON.stringify({ platform: process.platform, node: process.version, rounds }), + ) + } finally { + restore() + await rm(root, { recursive: true, force: true }) + } +}) diff --git a/src/main/services/project-instructions.test.ts b/src/main/services/project-instructions.test.ts index f0898c1d55..9c4b2a23f5 100644 --- a/src/main/services/project-instructions.test.ts +++ b/src/main/services/project-instructions.test.ts @@ -1,15 +1,20 @@ import { describe, it, beforeEach, afterEach } from 'node:test' import assert from 'node:assert/strict' -import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises' +import { mkdtemp, mkdir, writeFile, rm, symlink } from 'node:fs/promises' import { join } from 'node:path' import { tmpdir } from 'node:os' import { + activateNestedInstructionSources, + createNestedInstructionTurn, + invalidateNestedInstructionDiscoveryForWrite, loadInstructionLayers, + loadInstructionLayersWithMetadata, loadProjectInstructionSources, loadAgentRequestedRulesCatalog, } from './project-instructions.ts' import { setWorkspaceRootForTest } from './workspace.ts' import { runWithWorkspaceTrust } from './security/workspace-trust.ts' +import { runWithThreadExecutionContext } from './thread-execution-context.ts' describe('project-instructions', () => { let projectRoot = '' @@ -56,6 +61,7 @@ describe('project-instructions', () => { scope: 'project', content: 'Use tabs.', active: true, + trusted: true, }, ]) const layers = await withTrust(true, () => loadInstructionLayers()) @@ -156,6 +162,369 @@ describe('project-instructions', () => { assert.match(layers.project, / { + await writeFile(join(projectRoot, 'AGENTS.md'), 'Root rules') + await mkdir(join(projectRoot, 'packages', 'api', 'src'), { recursive: true }) + await mkdir(join(projectRoot, 'packages', 'web', 'src'), { recursive: true }) + await writeFile(join(projectRoot, 'packages', 'AGENTS.md'), 'Package rules') + await writeFile(join(projectRoot, 'packages', 'api', 'AGENTS.md'), 'API rules') + await writeFile(join(projectRoot, 'packages', 'web', 'AGENTS.md'), 'Web rules') + + const sources = await withTrust(true, () => + loadProjectInstructionSources({ nestedContextPaths: ['packages/api/src/server.ts'] }), + ) + assert.deepEqual( + sources + .filter((source) => source.scopePath !== undefined) + .map((source) => ({ + name: source.name, + scopePath: source.scopePath, + active: source.active, + })), + [ + { name: 'packages/AGENTS.md', scopePath: 'packages', active: true }, + { name: 'packages/api/AGENTS.md', scopePath: 'packages/api', active: true }, + { name: 'packages/web/AGENTS.md', scopePath: 'packages/web', active: false }, + ], + ) + + const layers = await withTrust(true, () => + loadInstructionLayers({ nestedContextPaths: ['packages/api/src/server.ts'] }), + ) + assert.ok(layers.project.indexOf('Root rules') < layers.project.indexOf('Package rules')) + assert.ok(layers.project.indexOf('Package rules') < layers.project.indexOf('API rules')) + assert.doesNotMatch(layers.project, /Web rules/) + assert.match(layers.project, /path="packages\/api\/AGENTS\.md"/) + }) + + it('deduplicates and orders multiple sibling targets deterministically', async () => { + for (const name of ['api', 'web']) { + await mkdir(join(projectRoot, 'packages', name), { recursive: true }) + await writeFile(join(projectRoot, 'packages', name, 'AGENTS.md'), `${name} rules`) + } + const build = (paths: string[]): Promise => + withTrust( + true, + async () => (await loadInstructionLayers({ nestedContextPaths: paths })).project, + ) + const forward = await build(['packages/api/a.ts', 'packages/web/b.ts', 'packages/api/a.ts']) + const reverse = await build(['packages/web/b.ts', 'packages/api/a.ts']) + assert.equal(forward, reverse) + assert.ok(forward.indexOf('api rules') < forward.indexOf('web rules')) + }) + + it('keeps nested AGENT.md and CLAUDE.md root-only', async () => { + await mkdir(join(projectRoot, 'packages', 'api'), { recursive: true }) + await writeFile(join(projectRoot, 'packages', 'api', 'AGENT.md'), 'Nested singular') + await writeFile(join(projectRoot, 'packages', 'api', 'CLAUDE.md'), 'Nested Claude') + assert.deepEqual( + await withTrust(true, () => + loadProjectInstructionSources({ nestedContextPaths: ['packages/api/file.ts'] }), + ), + [], + ) + }) + + it('ignores generated trees and symlink escapes for discovery and activation', async () => { + const outside = await mkdtemp(join(tmpdir(), 'copse-panel-instructions-outside-')) + try { + await mkdir(join(projectRoot, 'packages', 'api'), { recursive: true }) + await mkdir(join(projectRoot, 'node_modules', 'dep'), { recursive: true }) + await mkdir(join(projectRoot, 'dist', 'generated'), { recursive: true }) + await writeFile(join(projectRoot, 'packages', 'api', 'AGENTS.md'), 'API rules') + await writeFile(join(projectRoot, 'node_modules', 'dep', 'AGENTS.md'), 'Dependency rules') + await writeFile(join(projectRoot, 'dist', 'generated', 'AGENTS.md'), 'Generated rules') + await writeFile(join(outside, 'AGENTS.md'), 'Outside rules') + await writeFile(join(outside, 'file.ts'), 'outside') + await symlink(join(outside, 'AGENTS.md'), join(projectRoot, 'packages', 'AGENTS.md')) + await symlink(outside, join(projectRoot, 'packages', 'api', 'outside')) + + const sources = await withTrust(true, () => + loadProjectInstructionSources({ + nestedContextPaths: ['packages/api/outside/file.ts'], + }), + ) + assert.deepEqual( + sources.map((source) => ({ name: source.name, active: source.active })), + [{ name: 'packages/api/AGENTS.md', active: false }], + ) + } finally { + await rm(outside, { recursive: true, force: true }) + } + }) + + it('bounds an unusually deep active chain while retaining the nearest rules', async () => { + const segments: string[] = [] + for (let depth = 1; depth <= 12; depth++) { + segments.push(`d${String(depth)}`) + const dir = join(projectRoot, ...segments) + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'AGENTS.md'), `rules-${String(depth)}`) + } + const target = `${segments.join('/')}/file.ts` + const sources = await withTrust(true, () => + loadProjectInstructionSources({ nestedContextPaths: [target] }), + ) + const active = sources.filter((source) => source.scopePath !== undefined && source.active) + assert.equal(active.length, 8) + assert.ok(active.some((source) => source.content === 'rules-12')) + assert.ok(!active.some((source) => source.content === 'rules-1')) + }) + + it('keeps later scopes inactive after the prompt-wide byte budget is exhausted', async () => { + const activePaths = new Set() + const activeContents = new Set() + let activeBytes = 0 + + for (let index = 0; index < 7; index++) { + const scope = `scope-${String(index)}` + await mkdir(join(projectRoot, scope), { recursive: true }) + await writeFile(join(projectRoot, scope, 'AGENTS.md'), `${scope}: ${'x'.repeat(10 * 1024)}`) + } + + await withTrust(true, async () => { + for (let index = 0; index < 7; index++) { + const activation = await activateNestedInstructionSources( + [`scope-${String(index)}/file.ts`], + activePaths, + activeContents, + activeBytes, + ) + for (const path of activation.activatedPaths) activePaths.add(path) + for (const content of activation.injectedContents) { + activeContents.add(content) + activeBytes += Buffer.byteLength(content, 'utf-8') + } + if (index < 6) { + assert.equal(activation.injectedPaths.length, 1) + } else { + assert.deepEqual(activation.activatedPaths, []) + assert.equal(activation.block, '') + } + } + }) + }) + + it('memoizes referenced scopes and invalidates them after an AGENTS.md write', async () => { + await mkdir(join(projectRoot, 'packages', 'api'), { recursive: true }) + await mkdir(join(projectRoot, 'packages', 'web'), { recursive: true }) + await writeFile(join(projectRoot, 'packages', 'api', 'AGENTS.md'), 'API rules') + const turn = createNestedInstructionTurn() + const activate = (paths: string[]): ReturnType => + withTrust(true, () => activateNestedInstructionSources(paths, new Set(), new Set(), 0, turn)) + + const first = await activate(['packages/api/a.ts']) + assert.deepEqual(first.injectedNames, ['packages/api/AGENTS.md']) + + // Cache a missing instruction when this scope is first referenced. + assert.deepEqual((await activate(['packages/web/b.ts'])).injectedNames, []) + await writeFile(join(projectRoot, 'packages', 'web', 'AGENTS.md'), 'Web rules') + const stale = await activate(['packages/web/b.ts']) + assert.deepEqual(stale.injectedNames, []) + + // A write to something other than an AGENTS.md keeps the memo… + assert.equal(invalidateNestedInstructionDiscoveryForWrite(['packages/web/b.ts'], turn), false) + assert.deepEqual((await activate(['packages/web/b.ts'])).injectedNames, []) + // …while a write to the AGENTS.md itself forgets it, so the next call sees the file. + assert.equal( + invalidateNestedInstructionDiscoveryForWrite(['packages/web/AGENTS.md'], turn), + true, + ) + assert.deepEqual((await activate(['packages/web/b.ts'])).injectedNames, [ + 'packages/web/AGENTS.md', + ]) + }) + + it('reads a newly referenced scope during a turn and refreshes known scopes next turn', async () => { + await mkdir(join(projectRoot, 'packages', 'api'), { recursive: true }) + const turn = createNestedInstructionTurn() + await withTrust(true, () => activateNestedInstructionSources([], new Set(), new Set(), 0, turn)) + const path = join(projectRoot, 'packages', 'api', 'AGENTS.md') + await writeFile(path, 'First rules') + const first = await withTrust(true, () => + activateNestedInstructionSources(['packages/api/file.ts'], new Set(), new Set(), 0, turn), + ) + assert.match(first.block, /First rules/) + await writeFile(path, 'Updated externally') + const next = await withTrust(true, () => + activateNestedInstructionSources( + ['packages/api/file.ts'], + new Set(), + new Set(), + 0, + createNestedInstructionTurn(), + ), + ) + assert.match(next.block, /Updated externally/) + }) + + it('keeps turn ancestor reads within checkout, generated-tree and symlink boundaries', async () => { + await mkdir(join(projectRoot, 'packages', 'nested', '.git'), { recursive: true }) + await mkdir(join(projectRoot, 'packages', 'node_modules', 'dep'), { recursive: true }) + await writeFile(join(projectRoot, 'packages', 'AGENTS.md'), 'Parent rules') + await writeFile(join(projectRoot, 'packages', 'nested', 'AGENTS.md'), 'Nested checkout rules') + await writeFile( + join(projectRoot, 'packages', 'node_modules', 'dep', 'AGENTS.md'), + 'Dependency rules', + ) + await symlink(join(projectRoot, 'packages'), join(projectRoot, 'linked')) + const turn = createNestedInstructionTurn() + const activate = (paths: string[]): ReturnType => + withTrust(true, () => activateNestedInstructionSources(paths, new Set(), new Set(), 0, turn)) + for (const path of ['packages/nested/file.ts', 'packages/node_modules/dep/file.ts']) { + const result = await activate([path]) + assert.deepEqual(result.injectedNames, ['packages/AGENTS.md']) + assert.doesNotMatch(result.block, /Nested checkout rules|Dependency rules/) + } + assert.equal((await activate(['linked/file.ts'])).block, '') + const untrusted = await withTrust(false, () => + activateNestedInstructionSources( + ['packages/file.ts'], + new Set(), + new Set(), + 0, + createNestedInstructionTurn(), + ), + ) + assert.equal(untrusted.block, '') + }) + + it('flags project sources when discovery stops at a cap', async () => { + // Depth is the cheapest cap to hit: one directory beyond the limit. + const segments: string[] = [] + for (let depth = 1; depth <= 17; depth++) segments.push(`d${String(depth)}`) + await mkdir(join(projectRoot, ...segments), { recursive: true }) + await writeFile(join(projectRoot, ...segments, 'AGENTS.md'), 'Too deep') + await writeFile(join(projectRoot, 'AGENTS.md'), 'Root rules') + + const sources = await withTrust(true, () => + loadProjectInstructionSources({ refreshNestedDiscovery: true }), + ) + assert.deepEqual( + sources.map((source) => ({ name: source.name, truncated: source.discoveryTruncated })), + [{ name: 'AGENTS.md', truncated: true }], + ) + + await rm(join(projectRoot, 'd1'), { recursive: true, force: true }) + const complete = await withTrust(true, () => + loadProjectInstructionSources({ refreshNestedDiscovery: true }), + ) + assert.equal(complete[0]?.discoveryTruncated, undefined) + }) + + it('lists a nested file that repeats the root rules as a duplicate, injected once', async () => { + await writeFile(join(projectRoot, 'AGENTS.md'), 'Shared rules') + await mkdir(join(projectRoot, 'packages', 'api'), { recursive: true }) + await writeFile(join(projectRoot, 'packages', 'api', 'AGENTS.md'), 'Shared rules') + + const sources = await withTrust(true, () => + loadProjectInstructionSources({ nestedContextPaths: ['packages/api/a.ts'] }), + ) + assert.deepEqual( + sources.map((source) => ({ name: source.name, duplicateOf: source.duplicateOf })), + [ + { name: 'AGENTS.md', duplicateOf: undefined }, + { name: 'packages/api/AGENTS.md', duplicateOf: 'AGENTS.md' }, + ], + ) + const layers = await withTrust(true, () => + loadInstructionLayers({ nestedContextPaths: ['packages/api/a.ts'] }), + ) + assert.equal(layers.project.match(/ { + t.mock.method(Date, 'now', () => 1_000) + for (const name of ['api', 'web']) { + await mkdir(join(projectRoot, 'packages', name), { recursive: true }) + await writeFile(join(projectRoot, 'packages', name, 'AGENTS.md'), `${name} rules`) + } + const inThread = (threadId: string, fn: () => Promise): Promise => + withTrust(true, () => + runWithThreadExecutionContext( + { + projectId: 'project', + threadId, + projectRoot, + root: projectRoot, + checkoutMode: 'shared', + branch: null, + }, + fn, + ), + ) + const activeNames = ( + sources: { name: string; active: boolean; scopePath?: string }[], + ): string[] => sources.filter((s) => s.scopePath !== undefined && s.active).map((s) => s.name) + + await inThread('thread-a', () => + loadInstructionLayersWithMetadata({ nestedContextPaths: ['packages/api/a.ts'] }, true), + ) + await inThread('thread-b', () => + loadInstructionLayersWithMetadata({ nestedContextPaths: ['packages/web/b.ts'] }, true), + ) + + const latest = { useLatestNestedActivation: true } + assert.deepEqual( + activeNames(await inThread('thread-a', () => loadProjectInstructionSources(latest))), + ['packages/api/AGENTS.md'], + ) + // Settings runs outside any thread and reads the most recent turn. + assert.deepEqual( + activeNames(await withTrust(true, () => loadProjectInstructionSources(latest))), + ['packages/web/AGENTS.md'], + ) + // Updating an existing thread must become latest even with the same clock tick. + await inThread('thread-a', () => + loadInstructionLayersWithMetadata({ nestedContextPaths: ['packages/api/a.ts'] }, true), + ) + assert.deepEqual( + activeNames(await withTrust(true, () => loadProjectInstructionSources(latest))), + ['packages/api/AGENTS.md'], + ) + }) + + it('reports worktree activation against the stable project root in Sources', async () => { + const worktreeRoot = await mkdtemp(join(tmpdir(), 'copse-panel-instructions-worktree-')) + const projectAlias = `${projectRoot}-alias` + try { + await symlink(projectRoot, projectAlias, 'dir') + for (const root of [projectRoot, worktreeRoot]) { + await mkdir(join(root, 'packages', 'api'), { recursive: true }) + await writeFile(join(root, 'packages', 'api', 'AGENTS.md'), 'API worktree rules') + } + + await withTrust(true, () => + runWithThreadExecutionContext( + { + projectId: 'project', + threadId: 'thread', + projectRoot: projectAlias, + root: worktreeRoot, + checkoutMode: 'worktree', + branch: 'codex/thread', + }, + () => + loadInstructionLayersWithMetadata( + { nestedContextPaths: ['packages/api/file.ts'] }, + true, + ), + ), + ) + + const sources = await withTrust(true, () => + loadProjectInstructionSources({ + useLatestNestedActivation: true, + refreshNestedDiscovery: true, + }), + ) + assert.equal(sources.find((source) => source.name === 'packages/api/AGENTS.md')?.active, true) + } finally { + await rm(projectAlias, { force: true }) + await rm(worktreeRoot, { recursive: true, force: true }) + } + }) + it('gates Cursor rules with the same trust gate', async () => { await mkdir(join(projectRoot, '.cursor', 'rules'), { recursive: true }) await writeFile( diff --git a/src/main/services/project-instructions.ts b/src/main/services/project-instructions.ts index 9dff3d37ba..dd9d2e89c8 100644 --- a/src/main/services/project-instructions.ts +++ b/src/main/services/project-instructions.ts @@ -1,6 +1,6 @@ import * as fsp from 'node:fs/promises' import { homedir } from 'node:os' -import { join } from 'node:path' +import { basename, dirname, isAbsolute, join, relative, resolve, sep } from 'node:path' import type { InstructionScope } from '@shared/types/instructions.ts' import { buildAgentRequestedRulesCatalog, @@ -8,43 +8,172 @@ import { loadCursorRuleSources, type CursorRuleContext, } from './skills/cursor-rules.ts' -import { getWorkspaceRoot } from './workspace.ts' +import { getAgentExecutionRoot, getAgentProjectRoot } from './execution-root.ts' import { isWorkspaceTrusted } from './security/workspace-trust.ts' +import { getThreadExecutionContext } from './thread-execution-context.ts' /** * Project-root instruction files, in precedence order. * - * `AGENT.md` / `AGENTS.md` are the cross-tool convention; `CLAUDE.md` is Claude Code's. - * We load whichever are present so Copse behaves the same regardless of which assistant - * seeded the repo. Identical content (repos often symlink `AGENTS.md` → `CLAUDE.md`) is - * loaded once. + * Identical content is loaded once. Only `AGENTS.md` receives nested, + * directory-scoped semantics: `AGENT.md` and `CLAUDE.md` remain root-only + * compatibility formats. */ export const PROJECT_INSTRUCTION_FILES = ['AGENT.md', 'AGENTS.md', 'CLAUDE.md'] as const +/** User-global instruction files, relative to the home directory, in precedence order. */ +export const GLOBAL_INSTRUCTION_FILES = ['AGENTS.md', join('.claude', 'CLAUDE.md')] as const + +const NESTED_INSTRUCTION_FILE = 'AGENTS.md' +const MAX_NESTED_DISCOVERY_DEPTH = 16 +const MAX_NESTED_DISCOVERY_DIRECTORIES = 10_000 +const MAX_NESTED_DISCOVERED_FILES = 200 +const MAX_NESTED_CONTEXT_PATHS = 64 +const MAX_ACTIVE_NESTED_FILES = 8 +const MAX_NESTED_FILE_BYTES = 32 * 1024 +const MAX_ACTIVE_NESTED_BYTES = 64 * 1024 /** - * User-global instruction files, relative to the home directory, in precedence order. - * These form a lower-precedence layer beneath the project files — always-on personal - * steering that applies across every workspace, mirroring how most assistants layer a - * global file under the per-repo one. + * How long a full inventory serves callers outside a turn (Settings and + * context estimates). Running turns read only referenced ancestor scopes. */ -export const GLOBAL_INSTRUCTION_FILES = ['AGENTS.md', join('.claude', 'CLAUDE.md')] as const +const NESTED_DISCOVERY_CACHE_MS = 30_000 + +/** Generated, vendored, or cache directories that cannot own workspace guidance. */ +const NESTED_SKIP_DIRS = new Set([ + 'node_modules', + '.git', + 'dist', + 'dist-test', + 'dist-types', + 'dist-test-iso', + 'out', + 'build', + 'target', + 'vendor', + 'coverage', + '.next', + '.nuxt', + '.svelte-kit', + '.turbo', + '.cache', + '.venv', + 'venv', + '__pycache__', +]) export interface ProjectInstructionSource { /** Absolute path of the file on disk. */ path: string - /** Bare filename (e.g. `CLAUDE.md`), for display. */ + /** Workspace-relative display name, or the user-global relative path. */ name: string /** Whether the file is user-global or project-scoped. */ scope: InstructionScope /** Trimmed file contents. */ content: string + /** Whether the file feeds the current/most recently assembled turn prompt. */ + active: boolean + /** False for project text until the workspace trust gate is granted. */ + trusted: boolean + /** Workspace-relative directory governed by a nested AGENTS.md. */ + scopePath?: string /** - * Whether the file feeds the system prompt. Project-scoped files are inert - * until the workspace is trusted (context-provenance plan, Phase 2) — the - * same gate that keeps project MCP servers from spawning (#100). Discovered - * so the UI can surface them; excluded from the prompt while `false`. + * Name of the earlier source whose content this nested file repeats. The + * content is injected once, through that source; this entry stays listed so + * Sources does not silently hide a file the workspace ships. */ - active: boolean + duplicateOf?: string + /** + * True on project sources when the nested walk stopped at a directory, file, + * or depth cap, so the listed nested files may be incomplete. + */ + discoveryTruncated?: boolean +} + +interface NestedInstructionSource { + path: string + name: string + content: string + scopePath: string +} + +/** One walk of the execution root for nested AGENTS.md files. */ +export interface NestedInstructionDiscovery { + sources: NestedInstructionSource[] + /** Directories visited. */ + directories: number + /** The walk stopped at a cap before covering the whole tree. */ + truncated: boolean +} + +interface NestedDiscoveryState { + directories: number + sources: NestedInstructionSource[] + truncated: boolean +} + +interface NestedInstructionDirectory { + descend: boolean + source?: NestedInstructionSource +} + +/** + * Per-turn ancestor reads, shared by concurrent tools. A scope is read when + * first referenced, never by scanning unrelated subtrees. New turns start + * fresh; AGENTS.md writes invalidate the execution root during the turn. + */ +export interface NestedInstructionTurn { + readonly discoveries: Map>> +} + +export function createNestedInstructionTurn(): NestedInstructionTurn { + return { discoveries: new Map() } +} + +interface NestedActivationRecord { + names: ReadonlySet + sequence: number +} + +/** + * Most recently assembled real-turn activation, for Settings → Sources. + * + * Keyed by the stable project root, then by thread: concurrent turns on + * different threads of one project must not overwrite each other's record. + * Settings has no thread of its own, so it reads the most recently updated + * thread. Names are workspace-relative because a turn may execute in an + * isolated worktree while Settings reads the primary project; absolute + * execution-root paths would never compare equal across that boundary. + */ +const lastNestedActivationByProjectRoot = new Map>() +// Wall-clock ticks can tie or move backwards; assembly order defines the latest activation. +let nestedActivationSequence = 0 +const nestedDiscoveryCache = new Map< + string, + { expiresAt: number; discovery: NestedInstructionDiscovery } +>() + +function activationThreadKey(): string { + return getThreadExecutionContext()?.threadId ?? '' +} + +function latestNestedActivation(projectKey: string): ReadonlySet { + const byThread = lastNestedActivationByProjectRoot.get(projectKey) + if (!byThread) return new Set() + const threadKey = activationThreadKey() + const own = threadKey ? byThread.get(threadKey) : undefined + if (own) return own.names + let latest: NestedActivationRecord | undefined + for (const record of byThread.values()) { + if (!latest || record.sequence > latest.sequence) latest = record + } + return latest?.names ?? new Set() +} + +function recordNestedActivation(projectKey: string, names: ReadonlySet): void { + const byThread = + lastNestedActivationByProjectRoot.get(projectKey) ?? new Map() + byThread.set(activationThreadKey(), { names, sequence: ++nestedActivationSequence }) + lastNestedActivationByProjectRoot.set(projectKey, byThread) } async function readTrimmed(path: string): Promise { @@ -52,24 +181,374 @@ async function readTrimmed(path: string): Promise { const content = (await fsp.readFile(path, 'utf-8')).trim() return content || null } catch { - return null // missing file is normal + return null + } +} + +function isWithinRoot(path: string, root: string): boolean { + const rel = relative(root, path) + return rel === '' || (!isAbsolute(rel) && rel !== '..' && !rel.startsWith(`..${sep}`)) +} + +async function canonicalPath(path: string): Promise { + try { + return await fsp.realpath(path) + } catch { + return resolve(path) + } +} + +/** Read a project file only when its symlink target remains inside the workspace. */ +async function readProjectTrimmed( + path: string, + root: string, + maxBytes?: number, +): Promise { + try { + const [canonicalRoot, canonicalFile] = await Promise.all([ + fsp.realpath(root), + fsp.realpath(path), + ]) + if (!isWithinRoot(canonicalFile, canonicalRoot)) return null + if (maxBytes === undefined) return await readTrimmed(canonicalFile) + + const handle = await fsp.open(canonicalFile, 'r') + try { + const buffer = Buffer.alloc(maxBytes + 1) + const { bytesRead } = await handle.read(buffer, 0, buffer.length, 0) + const truncated = bytesRead > maxBytes + const content = buffer.subarray(0, Math.min(bytesRead, maxBytes)).toString('utf-8').trim() + if (!content) return null + return truncated + ? `${content}\n\n[Copse truncated this nested AGENTS.md at ${String(maxBytes)} bytes.]` + : content + } finally { + await handle.close() + } + } catch { + return null + } +} + +function displayPath(root: string, path: string): string { + return relative(root, path).split(sep).join('/') +} + +async function walkNestedInstructionFiles( + root: string, + dir: string, + depth: number, + state: NestedDiscoveryState, +): Promise { + if ( + depth > MAX_NESTED_DISCOVERY_DEPTH || + state.directories >= MAX_NESTED_DISCOVERY_DIRECTORIES || + state.sources.length >= MAX_NESTED_DISCOVERED_FILES + ) { + state.truncated = true + return + } + state.directories += 1 + + let entries + try { + entries = await fsp.readdir(dir, { withFileTypes: true }) + } catch { + return + } + entries.sort((a, b) => a.name.localeCompare(b.name)) + + // A nested checkout/submodule owns its own instructions. `.git` can be a file + // in a worktree, so match by name rather than only directory type. + if (depth > 0 && entries.some((entry) => entry.name === '.git')) return + + if (depth > 0) { + const instruction = entries.find((entry) => entry.name === NESTED_INSTRUCTION_FILE) + if (instruction?.isFile() || instruction?.isSymbolicLink()) { + const path = join(dir, instruction.name) + const content = await readProjectTrimmed(path, root, MAX_NESTED_FILE_BYTES) + if (content) { + state.sources.push({ + path, + name: displayPath(root, path), + content, + scopePath: displayPath(root, dir), + }) + } + } + } + + for (const entry of entries) { + if (!entry.isDirectory() || NESTED_SKIP_DIRS.has(entry.name)) continue + if ( + state.directories >= MAX_NESTED_DISCOVERY_DIRECTORIES || + state.sources.length >= MAX_NESTED_DISCOVERED_FILES + ) { + state.truncated = true + break + } + await walkNestedInstructionFiles(root, join(dir, entry.name), depth + 1, state) + } +} + +async function walkNestedInstructionTree( + key: string, + root: string, +): Promise { + const state: NestedDiscoveryState = { directories: 0, sources: [], truncated: false } + await walkNestedInstructionFiles(root, root, 0, state) + const discovery: NestedInstructionDiscovery = { + sources: state.sources.sort((a, b) => a.name.localeCompare(b.name)), + directories: state.directories, + truncated: state.truncated, + } + if (discovery.truncated) { + console.warn( + `[instructions] nested AGENTS.md discovery under ${root} stopped early ` + + `(${String(discovery.directories)} directories, ${String(discovery.sources.length)} files): ` + + 'nested instruction files beyond the cap are not loaded.', + ) + } + nestedDiscoveryCache.set(key, { expiresAt: Date.now() + NESTED_DISCOVERY_CACHE_MS, discovery }) + return discovery +} + +interface NestedDiscoveryOptions { + /** Read referenced ancestor scopes once per turn. */ + turn?: NestedInstructionTurn | undefined + contextPaths?: readonly string[] | undefined + /** Explicit reload (Settings → Sources): ignore any cached walk. */ + refresh?: boolean | undefined +} + +async function discoverNestedInstructionSources( + root: string, + opts: NestedDiscoveryOptions = {}, +): Promise { + const key = resolve(root) + if (opts.turn) { + let scopes = opts.turn.discoveries.get(key) + if (!scopes) { + scopes = new Map() + opts.turn.discoveries.set(key, scopes) + } + return discoverReferencedAncestors(root, opts.contextPaths ?? [], scopes) + } + const cached = nestedDiscoveryCache.get(key) + if (!opts.refresh && cached && cached.expiresAt > Date.now()) return cached.discovery + return walkNestedInstructionTree(key, root) +} + +/** + * Forget the current execution root's discovery after the agent wrote, moved, + * or removed a nested AGENTS.md, so the next tool call re-reads its ancestors. + * Only file paths named AGENTS.md count; other writes keep the memo. Returns + * whether anything was invalidated. + */ +export function invalidateNestedInstructionDiscoveryForWrite( + writtenPaths: readonly string[], + turn?: NestedInstructionTurn, +): boolean { + if (!writtenPaths.some((path) => basename(path.trim()) === NESTED_INSTRUCTION_FILE)) return false + const root = getAgentExecutionRoot() + if (!root) return false + const key = resolve(root) + nestedDiscoveryCache.delete(key) + turn?.discoveries.delete(key) + return true +} + +/** Resolve a context path without letting a symlinked ancestor escape the workspace. */ +async function normalizeContextPathWithinRoot( + root: string, + rawPath: string, +): Promise { + const trimmed = rawPath.trim() + if (!trimmed || trimmed.includes('\0')) return null + const absolute = isAbsolute(trimmed) ? resolve(trimmed) : resolve(root, trimmed) + if (!isWithinRoot(absolute, resolve(root))) return null + + const canonicalRoot = await canonicalPath(root) + let existingAncestor = absolute + for (;;) { + try { + const canonicalAncestor = await fsp.realpath(existingAncestor) + if (!isWithinRoot(canonicalAncestor, canonicalRoot)) return null + break + } catch { + const parent = dirname(existingAncestor) + if (parent === existingAncestor) return null + existingAncestor = parent + } + } + + const rel = relative(resolve(root), absolute).split(sep).join('/') + return rel === '.' ? '' : rel +} + +/** Read one ancestor without following directory symlinks or crossing a checkout. */ +async function readInstructionDirectory( + root: string, + scopePath: string, +): Promise { + const dir = join(root, scopePath) + try { + if (!(await fsp.lstat(dir)).isDirectory()) return { descend: false } + try { + await fsp.lstat(join(dir, '.git')) + return { descend: false } + } catch (err) { + if (!(err instanceof Error && 'code' in err && err.code === 'ENOENT')) { + return { descend: false } + } + } + const path = join(dir, NESTED_INSTRUCTION_FILE) + const content = await readProjectTrimmed(path, root, MAX_NESTED_FILE_BYTES) + return content + ? { descend: true, source: { path, name: displayPath(root, path), content, scopePath } } + : { descend: true } + } catch { + return { descend: false } + } +} + +/** Turn latency depends on referenced path depth, not the size of the checkout. */ +async function discoverReferencedAncestors( + root: string, + contextPaths: readonly string[], + scopes: Map>, +): Promise { + const uniquePaths = [...new Set(contextPaths)] + let truncated = uniquePaths.length > MAX_NESTED_CONTEXT_PATHS + const sources = new Map() + await Promise.all( + uniquePaths.slice(0, MAX_NESTED_CONTEXT_PATHS).map(async (path) => { + const normalized = await normalizeContextPathWithinRoot(root, path) + if (!normalized) return + const segments = normalized.split('/') + const targetStat = await fsp.lstat(join(root, normalized)).catch(() => null) + if (!targetStat?.isDirectory()) segments.pop() + let scopePath = '' + for (const [index, segment] of segments.entries()) { + if (index >= MAX_NESTED_DISCOVERY_DEPTH) { + truncated = true + break + } + if (NESTED_SKIP_DIRS.has(segment)) break + scopePath = scopePath ? `${scopePath}/${segment}` : segment + let pending = scopes.get(scopePath) + if (!pending) { + if (scopes.size >= MAX_NESTED_DISCOVERY_DIRECTORIES) { + truncated = true + break + } + pending = readInstructionDirectory(root, scopePath) + scopes.set(scopePath, pending) + } + const directory = await pending + if (!directory.descend) break + if (directory.source) sources.set(directory.source.path, directory.source) + } + }), + ) + const ordered = [...sources.values()].sort((a, b) => a.name.localeCompare(b.name)) + return { + sources: ordered.slice(0, MAX_NESTED_DISCOVERED_FILES), + directories: scopes.size, + truncated: truncated || ordered.length > MAX_NESTED_DISCOVERED_FILES, + } +} + +function scopeDepth(scopePath: string): number { + return scopePath.split('/').filter(Boolean).length +} + +function instructionPrecedence(a: NestedInstructionSource, b: NestedInstructionSource): number { + return scopeDepth(a.scopePath) - scopeDepth(b.scopePath) || a.name.localeCompare(b.name) +} + +async function selectNestedInstructionSources( + root: string, + sources: NestedInstructionSource[], + contextPaths: readonly string[], +): Promise { + const normalized = await Promise.all( + [...new Set(contextPaths)] + .slice(0, MAX_NESTED_CONTEXT_PATHS) + .map((path) => normalizeContextPathWithinRoot(root, path)), + ) + const targets = normalized.filter((path) => path !== null) + if (targets.length === 0) return [] + + const applicable = sources + .filter((source) => + targets.some( + (target) => target === source.scopePath || target.startsWith(`${source.scopePath}/`), + ), + ) + .sort(instructionPrecedence) + + // Keep nearest rules under a pathological chain, then restore broad→narrow order. + const retained: NestedInstructionSource[] = [] + let retainedBytes = 0 + for (const source of [...applicable].reverse()) { + if (retained.length >= MAX_ACTIVE_NESTED_FILES) break + const bytes = Buffer.byteLength(source.content, 'utf-8') + if (retainedBytes + bytes > MAX_ACTIVE_NESTED_BYTES) continue + retained.push(source) + retainedBytes += bytes } + return retained.sort(instructionPrecedence) +} + +/** + * Identical content is injected once, through the earliest source in precedence + * order. A root or global duplicate is dropped from the list (the same text is + * already listed under its higher-precedence name); a nested duplicate stays + * listed and is marked, so Sources shows every directory-scoped file the + * workspace ships rather than hiding one because it repeats the root rules. + */ +function deduplicateSources(resolved: ProjectInstructionSource[]): ProjectInstructionSource[] { + const sources: ProjectInstructionSource[] = [] + const contentIndexes = new Map() + for (const source of resolved) { + const existingIndex = contentIndexes.get(source.content) + const existing = existingIndex === undefined ? undefined : sources[existingIndex] + if (existingIndex === undefined || !existing) { + contentIndexes.set(source.content, sources.length) + sources.push(source) + continue + } + if (source.scopePath === undefined) continue + // An inactive nested sibling must not shadow the identical nested source + // that matched this turn: the active copy carries the content, the other + // one is the duplicate. (A root or global copy is never inactive while a + // nested one is active — both sit behind the same trust gate.) + if (existing.scopePath !== undefined && !existing.active && source.active) { + sources[existingIndex] = { ...existing, duplicateOf: source.name } + contentIndexes.set(source.content, sources.length) + sources.push(source) + continue + } + sources.push({ ...source, duplicateOf: existing.name }) + } + return sources } export interface ProjectInstructionOptions { /** Turn context for Auto-Attached / Manual Cursor rules (issue #636). */ cursorRuleContext?: CursorRuleContext + /** Workspace paths relevant to this turn, used for nested AGENTS.md activation. */ + nestedContextPaths?: readonly string[] + /** Settings-only: report the most recently assembled real turn's activation. */ + useLatestNestedActivation?: boolean + /** Explicit Sources reload bypasses the cached discovery. */ + refreshNestedDiscovery?: boolean + /** The running turn's discovery memo; the turn-start walk seeds it. */ + nestedInstructionTurn?: NestedInstructionTurn } -/** - * Discover the instruction files feeding the system prompt, global layer first then - * project. Identical content is loaded once (across both layers), so a repo whose - * `AGENTS.md` matches the user's global file is not injected twice. - * - * Cursor rules: Always + legacy always; Auto-Attached when `cursorRuleContext` - * paths match; Manual when `@`-mentioned. Agent-Requested rules are catalogued - * separately via {@link loadAgentRequestedRulesCatalog}. - */ +/** Discover instruction sources, global layer first then project. */ export async function loadProjectInstructionSources( opts: ProjectInstructionOptions = {}, ): Promise { @@ -79,23 +558,72 @@ export async function loadProjectInstructionSources( for (const rel of GLOBAL_INSTRUCTION_FILES) { const path = join(home, rel) const content = await readTrimmed(path) - if (content) resolved.push({ path, name: rel, scope: 'global', content, active: true }) + if (content) { + resolved.push({ + path, + name: rel, + scope: 'global', + content, + active: true, + trusted: true, + }) + } } - const root = getWorkspaceRoot() - if (root) { - // Instruction text is one approval away from execution for an agent with - // run_shell, so a cloned repo's AGENTS.md gets the same trust gate as its - // .mcp.json: discovered and listed, but inert until the user trusts the - // workspace. - const trusted = isWorkspaceTrusted(root) + const root = getAgentExecutionRoot() + const projectRoot = getAgentProjectRoot() + if (root && projectRoot) { + const trusted = isWorkspaceTrusted(projectRoot) for (const name of PROJECT_INSTRUCTION_FILES) { const path = join(root, name) - const content = await readTrimmed(path) - if (content) resolved.push({ path, name, scope: 'project', content, active: trusted }) + const content = await readProjectTrimmed(path, root) + if (content) { + resolved.push({ + path, + name, + scope: 'project', + content, + active: trusted, + trusted, + }) + } + } + + const discovery = await discoverNestedInstructionSources(root, { + turn: opts.nestedInstructionTurn, + contextPaths: opts.nestedContextPaths, + refresh: opts.refreshNestedDiscovery, + }) + const nested = discovery.sources + const explicitlyActive = + opts.nestedContextPaths !== undefined + ? new Set( + (await selectNestedInstructionSources(root, nested, opts.nestedContextPaths)).map( + (source) => source.path, + ), + ) + : null + const latestActive = opts.useLatestNestedActivation + ? latestNestedActivation(await canonicalPath(projectRoot)) + : new Set() + for (const source of nested) { + resolved.push({ + ...source, + scope: 'project', + active: trusted && (explicitlyActive?.has(source.path) ?? latestActive.has(source.name)), + trusted, + }) + } + if (discovery.truncated) { + // Every project row carries the flag, root files included: with no nested + // file found before the cap, they are the only rows that can say the list + // is short. + for (const source of resolved) { + if (source.scope === 'project') source.discoveryTruncated = true + } } - // Cursor project rules (`.cursor/rules/*.mdc` + legacy `.cursorrules`) — project text, - // applied after the top-level instruction files. + + // Cursor project rules are applied after AGENTS.md layers. for (const rule of await loadCursorRuleSources(root, opts.cursorRuleContext ?? {})) { resolved.push({ path: rule.path, @@ -103,42 +631,29 @@ export async function loadProjectInstructionSources( scope: 'project', content: rule.content, active: trusted, + trusted, }) } } - // De-duplicate identical content across files and scopes (e.g. a repo whose `AGENTS.md` - // matches the user's global file, or a rule copied into `AGENT.md`). - const sources: ProjectInstructionSource[] = [] - const seenContent = new Set() - for (const source of resolved) { - if (seenContent.has(source.content)) continue - seenContent.add(source.content) - sources.push(source) - } - return sources + return deduplicateSources(resolved) } -/** The two instruction layers the system prompt places separately. */ export interface InstructionLayers { - /** - * Workspace-authored block: provenance guidance plus one - * `` envelope per active project source. When the - * workspace is untrusted this is instead a short Copse-authored note naming - * the inert files, so the agent can explain why they are not applied. - * Empty when the workspace has no instruction files at all. - */ project: string - /** User-global instruction text, joined plainly — the user keeps the last word. */ global: string } -/** - * Neutralise any opening or closing `project_instructions` tag inside a source - * body so instruction content cannot forge or terminate its own envelope. - * Only the `<` of an offending tag is entity-escaped; everything else passes - * through verbatim. Same pattern as the external-content tool-result envelope. - */ +export interface InstructionLayerMetadata { + activeNestedPaths: string[] + activeInstructionContents: string[] + activeNestedBytes: number +} + +export interface InstructionLayersWithMetadata extends InstructionLayers { + metadata: InstructionLayerMetadata +} + function escapeInstructionTag(text: string): string { return text.replace(/<(?=\s*\/?\s*project_instructions)/gi, '<') } @@ -147,8 +662,6 @@ function escapeAttr(value: string): string { return value.replace(/&/g, '&').replace(/"/g, '"').replace(/ - `\n` + - `${escapeInstructionTag(s.content)}\n`, + (source) => + `\n` + + `${escapeInstructionTag(source.content)}\n`, ) .join('\n\n') return `## Workspace instructions\n\n${WORKSPACE_INSTRUCTIONS_GUIDANCE}\n\n${envelopes}` @@ -177,34 +690,159 @@ function buildGatedNote(names: string[]): string { ) } -/** - * Instruction layers for the system prompt. The project layer is wrapped in - * provenance envelopes and placed early (demoted below Copse steering); the - * global layer stays plain and is appended last. - */ -export async function loadInstructionLayers( +/** Build prompt layers and expose the active set to the local runtime. */ +export async function loadInstructionLayersWithMetadata( opts: ProjectInstructionOptions = {}, -): Promise { + trackActivation = false, +): Promise { const sources = await loadProjectInstructionSources(opts) const global = sources - .filter((s) => s.scope === 'global') - .map((s) => s.content) + .filter((source) => source.scope === 'global') + .map((source) => source.content) .join('\n\n') - const project = sources.filter((s) => s.scope === 'project') - const activeProject = project.filter((s) => s.active) - if (activeProject.length > 0) return { project: buildProjectBlock(activeProject), global } - if (project.length > 0) return { project: buildGatedNote(project.map((s) => s.name)), global } - return { project: '', global } + const project = sources.filter((source) => source.scope === 'project') + // A marked duplicate is listed, not injected: its content already rides on + // the source it duplicates. + const activeProject = project.filter( + (source) => source.active && source.duplicateOf === undefined, + ) + const activeNested = activeProject.filter((source) => source.scopePath !== undefined) + const activeNestedPaths = activeNested.map((source) => source.path) + + const root = getAgentExecutionRoot() + const projectRoot = getAgentProjectRoot() + if (trackActivation && root && projectRoot) { + recordNestedActivation( + await canonicalPath(projectRoot), + new Set( + project + .filter((source) => source.active && source.scopePath !== undefined) + .map((source) => source.name), + ), + ) + } + + const metadata: InstructionLayerMetadata = { + activeNestedPaths, + activeInstructionContents: sources + .filter((source) => source.active) + .map((source) => source.content), + activeNestedBytes: activeNested.reduce( + (total, source) => total + Buffer.byteLength(source.content, 'utf-8'), + 0, + ), + } + if (activeProject.length > 0) { + return { project: buildProjectBlock(activeProject), global, metadata } + } + if (project.length > 0) { + return { project: buildGatedNote(project.map((source) => source.name)), global, metadata } + } + return { project: '', global, metadata } +} + +/** Backwards-compatible prompt-only view used by tests and non-runtime callers. */ +export async function loadInstructionLayers( + opts: ProjectInstructionOptions = {}, +): Promise { + const { project, global } = await loadInstructionLayersWithMetadata(opts) + return { project, global } +} + +export interface NestedInstructionActivation { + block: string + activatedPaths: string[] + injectedPaths: string[] + /** Workspace-relative names of `injectedPaths`, for the transcript notice. */ + injectedNames: string[] + injectedContents: string[] +} + +const NO_ACTIVATION: NestedInstructionActivation = { + block: '', + activatedPaths: [], + injectedPaths: [], + injectedNames: [], + injectedContents: [], } /** - * Agent-requested Cursor rules catalog for the system prompt (empty when none). - * Trust-gated like the rules themselves — an untrusted repo does not get to - * advertise files for the agent to go read. + * Activate instructions for a file tool that introduced a path after turn + * start. The turn memo shares ancestor reads, including missing scopes; + * without one the shared inventory cache serves the request. */ +export async function activateNestedInstructionSources( + contextPaths: readonly string[], + alreadyActivePaths: ReadonlySet, + alreadyActiveContents: ReadonlySet, + alreadyActiveNestedBytes: number, + turn?: NestedInstructionTurn, +): Promise { + const root = getAgentExecutionRoot() + const projectRoot = getAgentProjectRoot() + if (!root || !projectRoot || !isWorkspaceTrusted(projectRoot)) return NO_ACTIVATION + + const selected = await selectNestedInstructionSources( + root, + (await discoverNestedInstructionSources(root, { turn, contextPaths })).sources, + contextPaths, + ) + const candidates = selected.filter( + (source) => !alreadyActivePaths.has(source.path) && !alreadyActiveContents.has(source.content), + ) + const fresh: NestedInstructionSource[] = [] + let remainingFiles = Math.max(0, MAX_ACTIVE_NESTED_FILES - alreadyActivePaths.size) + let remainingBytes = Math.max(0, MAX_ACTIVE_NESTED_BYTES - alreadyActiveNestedBytes) + for (const source of [...candidates].reverse()) { + const bytes = Buffer.byteLength(source.content, 'utf-8') + if (remainingFiles <= 0 || bytes > remainingBytes) continue + fresh.push(source) + remainingFiles -= 1 + remainingBytes -= bytes + } + fresh.sort(instructionPrecedence) + const injectedPaths = fresh.map((source) => source.path) + const injectedPathSet = new Set(injectedPaths) + // Sources skipped by the prompt-wide caps stay scoped rather than being + // reported as active. Identical content that is already present does count + // as active without consuming the budget twice. + const activatedPaths = selected + .filter( + (source) => + alreadyActivePaths.has(source.path) || + alreadyActiveContents.has(source.content) || + injectedPathSet.has(source.path), + ) + .map((source) => source.path) + const activatedPathSet = new Set(activatedPaths) + const projectKey = await canonicalPath(projectRoot) + const latest = new Set(latestNestedActivation(projectKey)) + for (const source of selected) { + if (activatedPathSet.has(source.path)) latest.add(source.name) + } + recordNestedActivation(projectKey, latest) + + if (fresh.length === 0) return { ...NO_ACTIVATION, activatedPaths } + const promptSources: ProjectInstructionSource[] = fresh.map((source) => ({ + ...source, + scope: 'project', + active: true, + trusted: true, + })) + return { + block: buildProjectBlock(promptSources), + activatedPaths, + injectedPaths, + injectedNames: fresh.map((source) => source.name), + injectedContents: fresh.map((source) => source.content), + } +} + +/** Agent-requested Cursor rules catalog for the system prompt (empty when none). */ export async function loadAgentRequestedRulesCatalog(): Promise { - const root = getWorkspaceRoot() - if (!root || !isWorkspaceTrusted(root)) return '' + const root = getAgentExecutionRoot() + const projectRoot = getAgentProjectRoot() + if (!root || !projectRoot || !isWorkspaceTrusted(projectRoot)) return '' const rules = await discoverCursorRules(root) return buildAgentRequestedRulesCatalog(rules) } diff --git a/src/renderer/views/settings-dialog.ts b/src/renderer/views/settings-dialog.ts index 63f00b3937..6e7bd501c4 100644 --- a/src/renderer/views/settings-dialog.ts +++ b/src/renderer/views/settings-dialog.ts @@ -1024,13 +1024,16 @@ export function mountSettingsDialog(store: AppStore, api: ApiClient): void {
Instruction files

- Files appended to the system prompt, in precedence order. Global steering + Files available to the system prompt, in precedence order. Global steering (~/AGENTS.md, ~/.claude/CLAUDE.md) loads first, then project AGENT.md/AGENTS.md (cross-tool), - CLAUDE.md (Claude Code), and always-applied Cursor rules + CLAUDE.md (Claude Code), directory-scoped nested + AGENTS.md, and always-applied Cursor rules (.cursor/rules/*.mdc with alwaysApply: true, plus .cursorrules). Auto-attached and manually @-mentioned - rules also join this list for the turn that activates them. + rules also join this list for the turn that activates them. Nested + AGENT.md and CLAUDE.md remain root-only compatibility + formats.

Loading… @@ -2843,15 +2846,38 @@ export function mountSettingsDialog(store: AppStore, api: ApiClient): void { * fix for "discovered but not loaded" sits on the thing reporting it. */ function makeInstructionRow(file: ProjectInstructionSummary): HTMLElement { + const nestedStatus = + file.scopePath === undefined + ? '' + : file.duplicateOf !== undefined + ? ` · scope: ${file.scopePath}/ · identical to ${file.duplicateOf}, loaded once through it` + : file.active + ? ` · scope: ${file.scopePath}/ · active this turn` + : ` · scope: ${file.scopePath}/ · activates when a path under this directory enters context` const detail = `${file.path} · ${formatByteSize(file.bytes)}` + - (file.active ? '' : ' · inert until you trust this workspace — click the badge to trust it') - const row = makeSourceRow(file.name, file.active ? file.scope : 'not loaded', detail, { - badgeClass: !file.active + (file.trusted + ? nestedStatus + : ' · inert until you trust this workspace — click the badge to trust it') + const badge = !file.trusted + ? 'not loaded' + : file.duplicateOf !== undefined + ? 'duplicate' + : file.scopePath !== undefined + ? file.active + ? 'active' + : 'scoped' + : file.scope + const row = makeSourceRow(file.name, badge, detail, { + badgeClass: !file.trusted ? 'sources-badge-untrusted' - : file.scope === 'project' - ? 'sources-badge-project' - : undefined, + : file.scopePath !== undefined + ? file.active && file.duplicateOf === undefined + ? 'sources-badge-auto' + : undefined + : file.scope === 'project' + ? 'sources-badge-project' + : undefined, titleAction: { label: `Open ${file.name}`, run: () => { @@ -2859,7 +2885,7 @@ export function mountSettingsDialog(store: AppStore, api: ApiClient): void { }, }, }) - if (file.active) return row + if (file.trusted) return row const badgeEl = row.querySelector('.sources-badge') if (badgeEl) { @@ -2892,8 +2918,18 @@ export function mountSettingsDialog(store: AppStore, api: ApiClient): void { fillSourceList( '#sources-instructions-list', instructions.map((f) => makeInstructionRow(f)), - 'No instruction files (add AGENT.md, AGENTS.md, or CLAUDE.md to the workspace root, or ~/AGENTS.md globally).', + 'No instruction files (add AGENT.md, AGENTS.md, or CLAUDE.md to the workspace root; nested directories may add AGENTS.md; or add ~/AGENTS.md globally).', ) + // Discovery is bounded; say so rather than let a missing nested file + // look like it was never written. + if (instructions.some((f) => f.discoveryTruncated)) { + const note = document.createElement('span') + note.className = 'sources-empty' + note.id = 'sources-instructions-truncated' + note.textContent = + 'Nested AGENTS.md discovery stopped at its directory limit, so this list may be incomplete. Deeper files are not loaded.' + qsRequired(overlay, '#sources-instructions-list').append(note) + } const kindLabel: Record = { always: 'always', diff --git a/src/renderer/views/settings-sources-instructions.test.ts b/src/renderer/views/settings-sources-instructions.test.ts index f9347b0fbd..2e4a499c98 100644 --- a/src/renderer/views/settings-sources-instructions.test.ts +++ b/src/renderer/views/settings-sources-instructions.test.ts @@ -24,6 +24,25 @@ function untrusted(): ProjectInstructionSummary { scope: 'project', bytes: 7129, active: false, + trusted: false, + } +} + +function nested( + name: string, + scopePath: string, + active: boolean, + extra: Partial = {}, +): ProjectInstructionSummary { + return { + path: `/workspace/${name}`, + name, + scope: 'project', + bytes: 256, + active, + trusted: true, + scopePath, + ...extra, } } @@ -60,7 +79,7 @@ function stubApi(initial: ProjectInstructionSummary[]): Harness { unsandboxedProjectHooks: () => Promise.resolve([]), setTrusted: (trusted: boolean) => { trustCalls.push(trusted) - files = files.map((file) => ({ ...file, active: trusted })) + files = files.map((file) => ({ ...file, active: trusted, trusted })) return Promise.resolve([]) }, }, @@ -109,6 +128,55 @@ describe('settings sources → instructions', () => { assert.doesNotMatch(detail, /7129 B/) }) + it('shows the latest turn activation separately from a nested file scope', async () => { + const list = await openInstructions( + stubApi([ + nested('packages/api/AGENTS.md', 'packages/api', true), + nested('packages/web/AGENTS.md', 'packages/web', false), + ]).api, + ) + const rows = [...list.querySelectorAll('.sources-row')] + const api = rows.find((row) => row.textContent.includes('packages/api/AGENTS.md')) + const web = rows.find((row) => row.textContent.includes('packages/web/AGENTS.md')) + assert.ok(api) + assert.ok(web) + assert.equal(api.querySelector('.sources-badge')?.textContent, 'active') + assert.match(api.querySelector('.sources-row-detail')?.textContent ?? '', /active this turn/) + assert.equal(web.querySelector('.sources-badge')?.textContent, 'scoped') + assert.match( + web.querySelector('.sources-row-detail')?.textContent ?? '', + /activates when a path under this directory enters context/, + ) + }) + + it('keeps a nested duplicate of the root rules listed, marked rather than hidden', async () => { + const list = await openInstructions( + stubApi([ + nested('packages/api/AGENTS.md', 'packages/api', true, { duplicateOf: 'AGENTS.md' }), + ]).api, + ) + const row = list.querySelector('.sources-row') + assert.ok(row) + assert.equal(row.querySelector('.sources-badge')?.textContent, 'duplicate') + assert.match( + row.querySelector('.sources-row-detail')?.textContent ?? '', + /identical to AGENTS\.md, loaded once through it/, + ) + assert.equal(list.querySelector('#sources-instructions-truncated'), null) + }) + + it('says when nested discovery stopped short instead of listing silently', async () => { + const list = await openInstructions( + stubApi([ + nested('packages/api/AGENTS.md', 'packages/api', false, { discoveryTruncated: true }), + ]).api, + ) + assert.match( + list.querySelector('#sources-instructions-truncated')?.textContent ?? '', + /this list may be incomplete/, + ) + }) + it('points at the badge as the way to trust an inert file', async () => { const list = await openInstructions(stubApi([untrusted()]).api) const badge = trustBadge(list) diff --git a/src/shared/api-protocol.mts b/src/shared/api-protocol.mts index 07d8d73bcd..7d3d541fe4 100644 --- a/src/shared/api-protocol.mts +++ b/src/shared/api-protocol.mts @@ -28,6 +28,6 @@ * doing on its own; until then a bump is the safe side of the disagreement, * since it can only refuse peers that would otherwise have been allowed. */ -// v4 adds bounded PR activity results. The whole-shape compatibility gate -// conservatively requires a bump for the optional activity payload. -export const API_PROTOCOL_VERSION = 4 as const +// v4 adds bounded PR activity results; v5 adds nested-instruction metadata. +// Both conservatively version optional result fields for the whole-shape gate. +export const API_PROTOCOL_VERSION = 5 as const diff --git a/src/shared/types/instructions.ts b/src/shared/types/instructions.ts index b94f972a03..66d766669e 100644 --- a/src/shared/types/instructions.ts +++ b/src/shared/types/instructions.ts @@ -11,6 +11,14 @@ export interface ProjectInstructionSummary { scope: InstructionScope /** Byte length of the trimmed content fed to the prompt. */ bytes: number - /** False when discovered but inert (project file in an untrusted workspace). */ + /** Whether the source was loaded into the current or most recently assembled prompt. */ active: boolean + /** Project trust gate, separate from conditional nested activation. */ + trusted: boolean + /** Workspace-relative directory governed by a nested AGENTS.md. */ + scopePath?: string + /** Name of the listed source this nested file repeats; its text is loaded once, via that one. */ + duplicateOf?: string + /** Nested discovery stopped at a cap, so nested files may be missing from the list. */ + discoveryTruncated?: boolean } diff --git a/tests/e2e/screenshots/settings-sources-nested-instructions.png b/tests/e2e/screenshots/settings-sources-nested-instructions.png new file mode 100644 index 0000000000..84f84c7130 Binary files /dev/null and b/tests/e2e/screenshots/settings-sources-nested-instructions.png differ diff --git a/tests/e2e/settings-sources-nested-instructions.e2e.ts b/tests/e2e/settings-sources-nested-instructions.e2e.ts new file mode 100644 index 0000000000..1c9e66cbb4 --- /dev/null +++ b/tests/e2e/settings-sources-nested-instructions.e2e.ts @@ -0,0 +1,124 @@ +import assert from 'node:assert/strict' +import { mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { $, browser, expect } from '@wdio/globals' +import { setComposerValue } from './helpers/composer.ts' +import { E2E_SCREENSHOT_DIR, saveElementScreenshot } from './helpers/screenshot.ts' +import { resetUserData, seedEmptyProject, writeSeedConfig } from './helpers/seed-config.ts' +import { waitForAgentIdle } from './helpers.ts' + +process.env['COPSE_PANEL_MOCK_LLM'] = '1' +process.env['ANTHROPIC_API_KEY'] = '' +process.env['OPENAI_API_KEY'] = '' + +const PROJECT_ID = 'e2e-settings-sources-nested-instructions' + +describe('settings sources nested AGENTS.md (#1354)', function () { + this.timeout(90_000) + let workspaceRoot = '' + + before(async () => { + mkdirSync(E2E_SCREENSHOT_DIR, { recursive: true }) + resetUserData() + workspaceRoot = mkdtempSync(join(tmpdir(), 'copse-e2e-nested-instructions-')) + mkdirSync(join(workspaceRoot, 'packages', 'api', 'src'), { recursive: true }) + mkdirSync(join(workspaceRoot, 'packages', 'web', 'src'), { recursive: true }) + mkdirSync(join(workspaceRoot, 'packages', 'duplicate'), { recursive: true }) + // Exceed the bounded inventory depth without adding hundreds of visible rows. + mkdirSync(join(workspaceRoot, ...Array.from({ length: 18 }, () => 'deep')), { recursive: true }) + writeFileSync( + join(workspaceRoot, 'packages', 'duplicate', 'AGENTS.md'), + 'Root workspace conventions.\n', + 'utf8', + ) + writeFileSync(join(workspaceRoot, 'AGENTS.md'), 'Root workspace conventions.\n', 'utf8') + writeFileSync( + join(workspaceRoot, 'packages', 'api', 'AGENTS.md'), + 'API package conventions.\n', + 'utf8', + ) + writeFileSync( + join(workspaceRoot, 'packages', 'web', 'AGENTS.md'), + 'Web package conventions.\n', + 'utf8', + ) + + seedEmptyProject(workspaceRoot, PROJECT_ID, { + subagentsEnabled: false, + model: 'claude-sonnet-4-6', + }) + writeSeedConfig({ + projects: [{ id: PROJECT_ID, path: workspaceRoot, name: 'workspace' }], + activeProjectId: PROJECT_ID, + trustedWorkspaceRoots: [realpathSync(workspaceRoot)], + [`threads:${PROJECT_ID}`]: [], + }) + await browser.reloadSession() + }) + + after(() => { + resetUserData() + rmSync(workspaceRoot, { recursive: true, force: true }) + }) + + it('shows active and inactive directory scopes after a path activates one branch', async () => { + await $('.prompt-input').waitForExist({ timeout: 30_000 }) + await setComposerValue('Review packages/api/src/router.ts and explain its role.') + await $('.submit-btn').click() + await $('.messages-list .msg-assistant').waitForExist({ timeout: 30_000 }) + await waitForAgentIdle(30_000) + + await $('[aria-label="Settings"]').click() + const dialog = $('#settings-dialog') + await expect(dialog).toBeDisplayed() + await dialog.$('button[data-section="customise"]').click() + const list = dialog.$('#sources-instructions-list') + await browser.waitUntil( + async () => { + const text = await list.getText() + return text.includes('packages/api/AGENTS.md') && text.includes('packages/web/AGENTS.md') + }, + { timeout: 15_000, timeoutMsg: 'expected both nested AGENTS.md sources' }, + ) + + const apiRow = list.$('.sources-row*=packages/api/AGENTS.md') + const webRow = list.$('.sources-row*=packages/web/AGENTS.md') + await expect(apiRow.$('.sources-badge')).toHaveText('active', { ignoreCase: true }) + await expect(webRow.$('.sources-badge')).toHaveText('scoped', { ignoreCase: true }) + const duplicateRow = list.$('.sources-row*=packages/duplicate/AGENTS.md') + await expect(duplicateRow.$('.sources-badge')).toHaveText('duplicate', { ignoreCase: true }) + assert.match( + await duplicateRow.$('.sources-row-detail').getText(), + /identical to AGENTS\.md, loaded once through it/, + ) + await expect($('#sources-instructions-truncated')).toHaveText( + expect.stringContaining('this list may be incomplete'), + ) + assert.match(await apiRow.$('.sources-row-detail').getText(), /scope: packages\/api\//) + assert.match(await apiRow.$('.sources-row-detail').getText(), /active this turn/) + assert.match( + await webRow.$('.sources-row-detail').getText(), + /activates when a path under this directory enters context/, + ) + + await browser.execute(() => { + const rows = document.querySelectorAll('#sources-instructions-list .sources-row') + for (const row of rows) { + const detail = row.querySelector('.sources-row-detail') + if (!detail?.textContent) continue + const parts = detail.textContent.split(' · ') + parts[0] = `/${row.querySelector('.sources-row-title')?.textContent ?? ''}` + detail.textContent = parts.join(' · ') + } + document + .querySelector('#sources-instructions-list') + ?.closest('fieldset') + ?.scrollIntoView({ block: 'start' }) + }) + await saveElementScreenshot( + 'fieldset:has(#sources-instructions-list)', + 'settings-sources-nested-instructions.png', + ) + }) +})