Skip to content

Avoid expensive incremental reconciliation after dependency paths move - #4865

Open
John Favret (johnfav03) wants to merge 2 commits into
microsoft:mainfrom
johnfav03:johnfav03/fix-incremental-pnpm-63731
Open

Avoid expensive incremental reconciliation after dependency paths move#4865
John Favret (johnfav03) wants to merge 2 commits into
microsoft:mainfrom
johnfav03:johnfav03/fix-incremental-pnpm-63731

Conversation

@johnfav03

Copy link
Copy Markdown
Contributor

Fixes microsoft/TypeScript#63731

When a dependency version change moves its resolved files to new paths, unchanged source files can have their reference targets replaced. The incremental builder previously reconciled this as an ordinary file change, which could lead it to compute a declaration shape signature and propagate possible shape changes through reverse dependencies. When this replacement impacts most of the program, reconciling is more expensive than a cold build - in the linked issue, the post-change incremental run performed more instantiations and took significantly longer than a cold build.

To fix this, we detect reference target replacement, and create a cold-build equivalent snapshot state when these conditions are met:

  • No existing non-default-library file changed structurally
  • A reference target was removed from the program
  • A new reference target was added
  • The reverse-dependency closure contains at least 8 checked files
  • That closure contains at least half of all checked files

Copilot AI balanced review requested due to automatic review settings August 10, 2026 16:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a heuristic to avoid costly incremental reconciliation when dependency paths move.

Changes:

  • Detects replaced reference targets and large reverse-dependency closures.
  • Resets eligible snapshots to cold-build state.
  • Adds a pnpm symlink regression scenario and baseline.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
internal/execute/incremental/programtosnapshot.go Implements detection and cold-state fallback.
internal/execute/tsctests/tsc_test.go Adds the incremental pnpm scenario.
internal/execute/tsctests/sys.go Adds a symlink-update test helper.
testdata/baselines/reference/tsc/incremental/pnpm-dependency-version-change-moves-resolved-paths.js Records expected incremental behavior.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread internal/execute/incremental/programtosnapshot.go
@jakebailey

Copy link
Copy Markdown
Member

Can you split this into two commits too?

This also seems a bit heuristic-y and a lot of code, not exactly sure how to make sure it's sound.

@johnfav03
John Favret (johnfav03) force-pushed the johnfav03/fix-incremental-pnpm-63731 branch from 8749a69 to a8d48ff Compare August 10, 2026 18:26
@johnfav03

Copy link
Copy Markdown
Contributor Author

Just split the changes into two commits. The difference is clear in Edit [0] in the changed baseline of the two commits; pre-fix, the baseline forces a .d.ts computation for shared.ts and version-signature updates for the replacement dependency and each route. Post-fix, the baseline is built from a cold-build state, and Signatures:: is empty, which shows that no signatures are being reconciled. In this linked issue, this reconciliation is responsible for work beyond a cold build, resulting in more instantiations and a substantially longer runtime.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

internal/execute/incremental/programtosnapshot.go:200

  • This early return leaves composite projects on the expensive reconciliation path even when every condition listed in the PR description is met. Composite projects can also resolve dependencies through moved pnpm paths, but the stated scope does not exclude them. Either support a cold-equivalent reset for composite snapshots, or explicitly narrow the PR's scope and add a rationale/test for why composite state cannot be reset safely.
	if t.oldProgram == nil || t.snapshot.options.Composite.IsTrue() {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--incremental: after a pnpm dependency version change, the cached run is slower than a cold run and most of the time is unattributed

3 participants