Skip to content

FE-879: Lazy per-slice cook worktrees and shared node_modules for brownfield#223

Open
kostandinang wants to merge 3 commits into
ka/fe-864-orchestrator-enhancementsfrom
ka/fe-879-lazy-cook-worktrees
Open

FE-879: Lazy per-slice cook worktrees and shared node_modules for brownfield#223
kostandinang wants to merge 3 commits into
ka/fe-864-orchestrator-enhancementsfrom
ka/fe-879-lazy-cook-worktrees

Conversation

@kostandinang

@kostandinang kostandinang commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Stack Context

Stacks on FE-864 (#212). Refines the brownfield per-slice worktree provisioning FE-755 introduced -- makes it lazy and shares node_modules -- without changing what cook produces. (brunch serve / FE-878 is what surfaced the startup cost.)

What?

  • Lazy provisioning -- slice-worktree creation moves out of the eager wireHandlers loop into resolveSliceCwd, via a new idempotent ensureSliceWorktree. A slice's worktree is materialized on first fire, so a run touching 2 of 8 slices pays for 2 worktrees, not 8. Rework re-fires are no-ops; provisioning is synchronous (execFileSync), so concurrent fires of distinct slices serialize on the JS thread -- no two git worktree add calls overlap under the parallel policy.
  • Shared node_modules -- each slice symlinks node_modules/ to the parent worktree's single copy instead of CoW-copying it per slice (SHAREABLE_TOP_LEVEL_ENTRIES). walkFiles already skips symlinks, so the shared tree is never re-walked during dependency seeding, merge, or promotion. Other gitignored dirs (dist/) still copy per slice.

Why?

In codebase mode the slice layout is always per-slice, and every slice's worktree was provisioned up front -- N git worktree add + N recursive node_modules CoW copies, paid synchronously before any slice fired (9 of each for an 8-slice plan). The per-slice node_modules copy dominates wall-clock.

Behavior / risk

Correctness-neutral: same worktrees on the same branches, just lazy; deps resolve through the symlink. Trade-off: build caches under node_modules (.cache, .vite) become shared across parallel slices -- fine for cook's transient runs; documented at the call site.

Tests

npm run verify green. New unit tests: slice node_modules is a symlink (not a copy), other gitignored content still copies, ensureSliceWorktree idempotent. Brownfield integration smoke unchanged.

kostandinang commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@kostandinang kostandinang changed the base branch from ka/fe-878-brunch-serve to graphite-base/223 June 16, 2026 14:17
@kostandinang kostandinang changed the base branch from graphite-base/223 to ka/fe-878-brunch-serve June 16, 2026 14:19
@kostandinang kostandinang marked this pull request as ready for review June 16, 2026 15:00
@cursor

cursor Bot commented Jun 16, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Behavior is intended to be correctness-neutral but changes when worktrees exist and how parallel slices share node_modules caches; collision handling on slice ids matching tracked paths is a new failure mode worth watching in brownfield runs.

Overview
Lazy slice provisioning (FE-879): Codebase mode no longer creates every slice worktree in wireHandlers before the run starts. Provisioning moves into resolveSliceCwd via new ensureSliceWorktree — first fire materializes the worktree; rework re-fires are idempotent no-ops. Runs that touch only a subset of slices pay for that many worktrees, not the full plan count.

Shared node_modules: copyMissingTopLevelEntries gains an optional symlink set; slice seeding passes SHAREABLE_TOP_LEVEL_ENTRIES (node_modules) so each slice links to the parent’s single install tree instead of a per-slice CoW copy. Other gitignored dirs (e.g. dist/) still copy per slice.

Tests / docs: New unit coverage for symlink behavior, ensureSliceWorktree idempotency and slice-id collision guards; memory/PLAN.md marks the cook over-copy follow-on resolved.

Reviewed by Cursor Bugbot for commit bda74ce. Bugbot is set up for automated code reviews on this repo. Configure here.

@kostandinang kostandinang changed the base branch from ka/fe-878-brunch-serve to graphite-base/223 June 16, 2026 15:31
@kostandinang kostandinang force-pushed the ka/fe-879-lazy-cook-worktrees branch from 3c0fa64 to e35fa92 Compare June 16, 2026 15:31
@kostandinang kostandinang changed the base branch from graphite-base/223 to ka/fe-864-orchestrator-enhancements June 16, 2026 15:31

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit e35fa92. Configure here.

Comment thread src/orchestrator/src/epic-sandbox-merge.ts
@kostandinang kostandinang changed the base branch from ka/fe-864-orchestrator-enhancements to graphite-base/223 June 16, 2026 23:44
@kostandinang kostandinang changed the base branch from graphite-base/223 to ka/fe-864-orchestrator-enhancements June 17, 2026 18:02
@kostandinang kostandinang self-assigned this Jun 18, 2026
kostandinang and others added 2 commits June 18, 2026 09:35
Brownfield cook provisioned every slice's git worktree eagerly in
wireHandlers — N `git worktree add` + N recursive node_modules CoW copies
paid synchronously at startup before any slice fired.

- Move slice-worktree creation into resolveSliceCwd via idempotent
  ensureSliceWorktree, so a slice's worktree is materialized on first fire.
  A run touching 2 of 8 slices pays for 2 worktrees, not 8. Synchronous
  provisioning serializes concurrent fires on the JS thread, so parallel-policy
  worktree adds never overlap.
- Symlink each slice's node_modules to the parent worktree's single copy
  instead of CoW-copying per slice (SHAREABLE_TOP_LEVEL_ENTRIES). walkFiles
  already skips symlinks, so the shared tree is never re-walked during dep
  seeding, merge, or promotion. Other gitignored dirs still copy per slice.

Correctness-neutral: same worktrees/branches, just lazy; deps resolve through
the symlink. npm run verify green; adds symlink + idempotency unit tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add the cook-worktree-laziness frontier definition (FE-879, done) and mark
the per-slice over-copy / eager-seeding optimization resolved — closing
cook-codebase-mode acceptance (8) and the sandcastle over-copy trigger.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@kostandinang kostandinang force-pushed the ka/fe-879-lazy-cook-worktrees branch from 8c8b78b to e7dcc42 Compare June 18, 2026 09:02
ensureSliceWorktree early-returned on existsSync, bypassing the collision
guard in seedSliceFromParentWorktree. A slice id matching a tracked parent
path (e.g. `src`) resolved to the project source dir, which cook then adopted
as the slice sandbox — silently breaking per-slice isolation. Only treat an
existing path as provisioned when it is a real git worktree (own `.git`).
@kostandinang kostandinang force-pushed the ka/fe-879-lazy-cook-worktrees branch from e7dcc42 to bda74ce Compare June 18, 2026 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant