Record materialized top-level outputs in completion values

[fmeum]

Description

This PR implements the consolidated design from @coeuvre's design document, replacing the dedicated download SkyFunction from earlier revisions of this PR: the materialization state of top-level outputs is recorded directly in the existing completion values instead of a child node, adding no Skyframe nodes or edges.

The commits are meant to be reviewed individually:

1. The invocation's download policy (mode, --remote_download_regex patterns, command) is injected as a precomputed value that completion functions depend on, so a policy change reevaluates them. This is a correction to the design document, which doesn't specify how completions are invalidated on policy changes after maybeInvalidateSkyframeValues is deleted; without it, e.g. a minimal build followed by a toplevel build of the same target would never download the top-level outputs. The policy must not be gated on Skymeld (non-Skymeld invocations also need policy-change invalidation, and SkymeldModule.beforeCommand runs after RemoteModule's).
2. TargetCompletionValue and AspectCompletionValue, previously empty singletons, record the outputs that are present in the local filesystem while their metadata tracked in Skyframe remains remote, together with their contents proxy. Invocations that materialize nothing keep sharing the empty singleton.
3. FilesystemValueChecker compares each completion value's record against the local filesystem and invalidates the completion node if a file is missing or modified; its reevaluation rematerializes the file if the policy wants it locally, without reexecuting or even revalidating the generating action. The completion functions become non-hermetic to permit this invalidation, matching action execution nodes. Lost-output rewinding stays in CompletionFunction, unchanged.
4. The previous-invocation state in RemoteOutputChecker (the lastRemoteOutputChecker chain and maybeInvalidateSkyframeValues) is deleted. The current invocation's policy check in shouldTrustMetadata stays for the non-Skymeld path.
5. The incremental build tests from Check previous build's toplevel outputs in shouldTrustMetadata #30852 are carried over; they fail on master and pass here. Fixes [7.6.1] Incremental build without any changes with BwoB within rules_xcodeproj is causing output artifacts to be redownloaded each time #26924.

The base commit is #30871, whose contents-proxy recording keeps the completion values' record precise for files that were verified by digest rather than freshly downloaded.

Two caveats relative to the design document, to be addressed separately: completion values are only produced on success, so partially materialized outputs of failed targets are still not tracked (the failed event path keeps its inline important output handler call); and the FSVC-notification-to-prefetcher-submission mechanism for --remote_download_regex-matched intermediate outputs (the document's second pillar) is left to a follow-up, including a story for lost remote blobs on that path.

Motivation

With Build without the Bytes, the completion functions download top-level outputs that only exist as remote metadata as an unmodeled side effect of their evaluation, and correctness across invocations was maintained by point mechanisms whose gaps surfaced as bugs: a top-level output deleted locally was silently not restored if any intervening invocation didn't request it (the new downloadToplevel_outputDeletedAfterUnrelatedBuild_toplevelOutputIsRestored test fails on master), and an invocation with a broader download policy caused the next incremental build to reevaluate and reexecute actions wholesale (#26924).

Full test runs: BuildWithoutTheBytesIntegrationTest (Java and shell), RemoteTests, FilesystemValueCheckerTest, RewindingTest, SkymeldBuildIntegrationTest, TargetCompleteEventTest.

Build API Changes

No

Checklist

• I have added tests for the new use cases (if any).
• I have updated the documentation (if applicable).

Release Notes

RELNOTES: With Build without the Bytes, locally materialized top-level outputs are now tracked in Skyframe. A requested top-level output that was deleted locally is reliably redownloaded in the next build that requests it, and changes to the download policy no longer cause unnecessary reevaluation or reexecution of actions.

[coeuvre]

I think solving the problem in skyframe is the right direction. I fed Gemini with the problem space and the PR, had some discussion and came up with another idea: artifact_level_materialization_design_doc.md.

[fmeum]

Thanks, I'll spend more time thinking through this analysis. My main concern is that one of the premises is wrong: Folks regularly do bazel test //... outside Google, which would add one SkyKey/SkyValue per output for all targets in the graph. I don't think that this is feasible without drastically increased memory usage.

We may need to revisit what top-level mode means for BwoB before we attempt this.

[coeuvre]

Yes, we don't want to increase the memory usage.

[coeuvre]

Asked Gemini to address the memory concern: consolidated_skyframe_materialization.md

"Instead of creating a fine-grained ArtifactMaterializationKey for every artifact—which would explode Skyframe memory on bazel test //...—the new design embeds top-level materialization directly into existing completion values with strictly zero new Skyframe nodes, handling intermediate regex downloads via FSVC-to-prefetcher notifications."