add replay build command as experimental - #3818
Conversation
github-actions
Bot
added
area/build
area/cli
area/docs
area/dependencies
area/tests
labels
194fce1 to
061e671
Compare
| if subjectPlat == nil || platforms.Only(*subjectPlat).Match(builderPlat) { | ||
| return platforms.Only(builderPlat) | ||
| } | ||
| return platforms.Any(*subjectPlat, builderPlat) |
There was a problem hiding this comment.
So with platforms.Any it matches both the subject and builder platforms but doesn't prefer the first one iiuc?
If a multi-platform material contains both, pickPlatformChild can archive whichever manifest appears first in the index, including the builder architecture instead of the subject architecture. Could this use platforms.Ordered(subject, builder) and have a test with both manifests in the index?
| if push { | ||
| var used bool | ||
| for _, e := range exports { | ||
| if e.Type == "image" { | ||
| if e.Attrs == nil { | ||
| e.Attrs = map[string]string{} | ||
| } | ||
| e.Attrs["push"] = "true" | ||
| if _, ok := e.Attrs["unpack"]; !ok { | ||
| e.Attrs["unpack"] = "false" | ||
| } | ||
| used = true | ||
| } | ||
| } | ||
| if !used { | ||
| exports = append(exports, &buildflags.ExportEntry{ | ||
| Type: "image", | ||
| Attrs: map[string]string{"push": "true", "unpack": "false"}, | ||
| }) | ||
| } | ||
| } |
There was a problem hiding this comment.
--output=type=registry is not normalized to image yet, so combining it with --push appends a second image exporter. That moight fail or push the same tag twice. Could the shorthand run after export normalization, as it does for regular build?
| if !explicit { | ||
| wantPlatforms = []ocispecs.Platform{platforms.Normalize(platforms.DefaultSpec())} | ||
| } | ||
|
|
||
| wantNames := make([]string, 0, len(wantPlatforms)) | ||
| matchers := make([]platforms.MatchComparer, 0, len(wantPlatforms)) | ||
| for _, platform := range wantPlatforms { | ||
| matchers = append(matchers, platforms.OnlyStrict(platform)) | ||
| wantNames = append(wantNames, platforms.Format(platform)) | ||
| } |
There was a problem hiding this comment.
This uses the CLI host OS to select a subject. On a Windows client connected to a Linux builder, a Linux-only multi-platform image has no match unless the user specifies --platform. Could the default come from the available subjects or builder instead?
| // Address the root directly by digest so we fetch the exact recorded | ||
| // index regardless of tag mutations since the original build. | ||
| fetchRef := ref | ||
| if !strings.Contains(fetchRef, "@") { | ||
| fetchRef = ref + "@" + rootDgst.String() | ||
| } | ||
| // Resolve the fetchRef first so we learn the root descriptor's size + | ||
| // mediaType. Without a size, contentutil.FromFetcher's ReaderAt reports | ||
| // size=0 and ReadBlob returns an empty payload (silently succeeding with | ||
| // a JSON decode error downstream). | ||
| _, rootDesc, err := imgResolver.Resolve(ctx, fetchRef) | ||
| if err != nil { | ||
| return ocispecs.Descriptor{}, nil, errors.Wrapf(err, "resolve image material %s", m.URI) | ||
| } | ||
| fetcher, err := imgResolver.Fetcher(ctx, fetchRef) | ||
| if err != nil { | ||
| return ocispecs.Descriptor{}, nil, errors.Wrapf(err, "fetch image material %s", m.URI) | ||
| } | ||
| provider := contentutil.FromFetcher(fetcher) | ||
| if rootDesc.Digest == "" { | ||
| rootDesc.Digest = rootDgst | ||
| } |
There was a problem hiding this comment.
If the purl already contains @sha256:X, this fetches X without checking it against rootDgst, the separate digest recorded in provenance. When they disagree, the snapshot can package a different material from the one recorded. Could snapshot reject that mismatch?
| cmd := buildxCmd(sb, withArgs( | ||
| "replay", "build", | ||
| "docker-image://"+ref, | ||
| "--platform=all", | ||
| "--dry-run", | ||
| )) |
There was a problem hiding this comment.
Thhat parses dry-run output as JSON, but replay build --dry-run defaults to the pretty format. Could it pass --format=json?
| var plan struct { | ||
| Subjects []struct { | ||
| Platform string `json:"platform"` | ||
| } `json:"subjects"` | ||
| } | ||
| require.NoError(t, json.Unmarshal(stdout.Bytes(), &plan), "dry-run must emit JSON plan") | ||
| require.Len(t, plan.Subjects, 2) |
There was a problem hiding this comment.
The plan puts the platform under subjects[].descriptor.platform iiuc, while this test decodes subjects[].platform. The unmarshalling ignores that missing field, so the test only checks the subject count. Could it check both descriptors?
| printer, err := progress.NewPrinter(printerCtx, dockerCli.Err(), "auto", | ||
| progress.WithDesc( | ||
| fmt.Sprintf("verifying %d subject(s) with %q instance using %s driver", len(req.Targets), b.Name, b.Driver), | ||
| fmt.Sprintf("%s:%s", b.Driver, b.Name), | ||
| ), | ||
| ) |
There was a problem hiding this comment.
It accepts --progress, but this printer always uses "auto" and the selected value never reaches VerifyRequest. Could verify pass through the requested mode?
c84885a to
a799871
Compare
tonistiigi
changed the title
Allow build.Options to select the frontend and pass frontend attributes directly instead of always solving with dockerfile.v0. Allow a PolicyConfig to carry a programmatic policy callback without a policy file. Callback-only entries run after file-based policies, require BuildKit's session source policy capability, and are not serialized. Both are needed by buildx replay, which replays the recorded frontend and pins sources with a policy callback. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Export ParseSLSAMaterial and AddPinToImage from the policy package and add imagetools.ReadProvenancePredicate to read the provenance predicate of an attestation manifest. Fetching referrers from an OCI layout that has none now returns an empty result instead of ErrNotFound, matching the registry behavior. Callers that treated ErrNotFound as "no referrers" behave the same. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add an experimental `buildx replay build` command that rebuilds an image from its SLSA provenance. By default, every source is pinned to the digest recorded in the provenance through a session source policy callback. --replay-mode=frontend replays the recorded frontend and options without pinning sources. Replay requires mode=max provenance, a Git or HTTP(S) build context and BuildKit v0.27 or later, and rebuilds one platform at a time. Subjects can be registry images, OCI layouts or attestation files. Sigstore signatures on the provenance are verified when present. --dry-run prints the replay plan and runs the same checks as a real replay. `replay snapshot` and `replay verify` are included as hidden commands until snapshots can be consumed by replay build and verification results can be trusted. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
- verify digests when reading the image index, attestation manifest and provenance, so replay uses the content covered by the signature - deny sources whose URI is not recorded in the provenance - validate every line of .intoto.jsonl inputs and require in-toto statements - reject stdin, OCI layout, local and Bake target contexts, local mode=delete outputs, --platform all, and recorded --network=host up front - keep a recorded --network=none unless --network is set - build the dry-run plan from the actual build options and validate outputs in dry-run - error on a policy config with both a callback and policy files - add Git context and multi-platform replay integration tests Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
a799871 to
278c48c
Compare
|
I cleaned up the current work so we should be able to get it to v0.38 . Only |
2 participants
first milestone for #3803