Pack eng/docker-tools content in ImageBuilder

AGENTS.md

@@ -10,8 +10,7 @@ The primary tool is ImageBuilder, a .NET CLI app that orchestrates builds from m
 | `src/ImageBuilder/` | ImageBuilder CLI and commands |
 | `src/ImageBuilder.Models/` | Manifest and image metadata models |
 | `src/ImageBuilder.Tests/` | MSTest, Moq, and Shouldly tests |
-| `eng/src/file-pusher/` | Storage file-pushing utility |
-| `eng/src/yaml-updater/` | YAML update utility |
+| `src/ImageBuilder.Updater/` | ImageBuilder infrastructure update PR utility |
 | `eng/docker-tools/` | Shared scripts and Azure Pipelines templates synchronized to other .NET Docker repositories |
 
 ImageBuilder commands inherit from `Command<TOptions>` and use System.CommandLine.
@@ -56,6 +55,16 @@ For combined development validation, use the engineering validation unofficial p
 in `eng/pipelines/dotnet-docker-tools-eng-validation-unofficial.yml` with the parameter
 `bootstrapImageBuilder: true`; each job then builds and uses ImageBuilder from the current source.
 
+## Bundled infrastructure
+
+ImageBuilder embeds `eng/docker-tools/` under `src/Infrastructure/Content/` so source and
+pipeline-template changes can be developed together. The copy lives under `src/` because that is
+the ImageBuilder container build context.
+
+The copies intentionally differ between releases: update `src/Infrastructure/Content/` for content
+the next ImageBuilder will ship; automation refreshes `eng/docker-tools/` when the repository
+adopts that ImageBuilder version. See `src/Infrastructure/README.md`.
+
 ## Documentation
 
 Update only the narrowest documentation affected by the change:

Microsoft.DotNet.DockerTools.slnx

@@ -4,11 +4,11 @@
     <Platform Name="x64" />
     <Platform Name="x86" />
   </Configurations>
-  <Project Path="eng/src/file-pusher/file-pusher.csproj" />
-  <Project Path="eng/src/yaml-updater/yaml-updater.csproj" />
   <Project Path="src/ImageBuilder/Microsoft.DotNet.ImageBuilder.csproj" />
+  <Project Path="src/ImageBuilder.Updater/ImageBuilder.Updater.csproj" />
   <Project Path="src/GitAutomation/Microsoft.DotNet.GitAutomation.csproj" />
   <Project Path="src/GitAutomation.Tests/Microsoft.DotNet.GitAutomation.Tests.csproj" />
   <Project Path="src/ImageBuilder.Models/Microsoft.DotNet.ImageBuilder.Models.csproj" />
   <Project Path="src/ImageBuilder.Tests/Microsoft.DotNet.ImageBuilder.Tests.csproj" />
+  <Project Path="src/Infrastructure/Microsoft.DotNet.DockerTools.Infrastructure.csproj" />
 </Solution>

eng/docker-tools/Update-ImageBuilder.ps1

@@ -0,0 +1,89 @@
+#!/usr/bin/env pwsh
+
+<#
+.SYNOPSIS
+Example script that updates the bundled docker-tools infrastructure in the current repo to a specific
+ImageBuilder image.
+
+.DESCRIPTION
+ImageBuilder ships a copy of the eng/docker-tools infrastructure and writes it back to disk via its
+'update' command. The reference that 'update' records into
+eng/docker-tools/templates/variables/docker-images.yml is supplied as an argument rather than being
+baked into the build, so the caller decides exactly which image the repo should pin to.
+
+This example resolves the multi-platform (manifest list / image index) digest of an ImageBuilder image
+(the published 'latest' tag by default) and passes that digest reference to the 'update' command, which
+runs inside the same image with the repository mounted so it can rewrite eng/docker-tools on disk.
+
+.PARAMETER ImageBuilderImage
+The ImageBuilder image to resolve and run. Defaults to the published 'latest' tag.
+
+.PARAMETER RepoRoot
+The root of the git repository to update. Defaults to the current directory.
+
+.NOTES
+To exercise an unpublished ImageBuilder (for example, the 'update' command before it is released),
+build the image, push it to a registry it can be pulled from, and pass its reference via
+-ImageBuilderImage. The digest is read from the registry, so the image must be pushed first.
+#>
+[CmdletBinding()]
+param(
+    [string]
+    $ImageBuilderImage = "mcr.microsoft.com/dotnet-buildtools/image-builder:latest",
+
+    [string]
+    $RepoRoot = (Get-Location).Path
+)
+
+Set-StrictMode -Version Latest
+$ErrorActionPreference = 'Stop'
+
+function Exec {
+    param ([string] $Cmd)
+
+    Write-Output "Executing: '$Cmd'"
+    Invoke-Expression $Cmd
+    if ($LASTEXITCODE -ne 0) {
+        throw "Failed: '$Cmd'"
+    }
+}
+
+# Strip any existing tag or digest so the resolved digest can be appended to the bare repository name.
+# A tag is a ':' within the final path segment; a registry host's ':port' precedes the last '/', so it
+# must not be mistaken for a tag.
+function Get-RepositoryName {
+    param ([string] $Reference)
+
+    $withoutDigest = $Reference.Split('@')[0]
+    $lastSlash = $withoutDigest.LastIndexOf('/')
+    $lastColon = $withoutDigest.LastIndexOf(':')
+    if ($lastColon -gt $lastSlash) {
+        return $withoutDigest.Substring(0, $lastColon)
+    }
+
+    return $withoutDigest
+}
+
+# Resolve the multi-platform digest. 'docker buildx imagetools inspect' reads the top-level manifest
+# straight from the registry, so for a multi-arch image this is the manifest list (image index) digest
+# rather than a single platform's digest. Pinning the index keeps the reference valid on every
+# platform the pipeline runs on.
+$digest = (docker buildx imagetools inspect $ImageBuilderImage --format '{{.Manifest.Digest}}')
+if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($digest)) {
+    throw "Unable to resolve a multi-platform digest for '$ImageBuilderImage'."
+}
+
+$repository = Get-RepositoryName $ImageBuilderImage
+$imageBuilderRef = "$repository@$($digest.Trim())"
+
+Write-Output "Resolved ImageBuilder reference: $imageBuilderRef"
+
+# Run 'update' from the resolved digest, mounting the repository so it can write eng/docker-tools to
+# disk. The command must run from the repository root, which is why $RepoRoot is the mounted working
+# directory. Running by the same digest that gets recorded keeps the writer and the pinned reference
+# identical.
+Exec ("docker run --rm " `
+    + "-v `"${RepoRoot}:/repo`" " `
+    + "-w /repo " `
+    + "$imageBuilderRef " `
+    + "update $imageBuilderRef")

eng/pipelines/update-image-builder-tag.yml

@@ -22,24 +22,48 @@ jobs:
   pool:
     vmImage: $(defaultLinuxAmd64PoolImage)
   steps:
-  - download: image-builder
-    artifact: source-build-id
-    displayName: Download Source Build ID artifact
-  - script: >
-      echo "##vso[task.setvariable variable=imageBuilderTag]$(cat $(Pipeline.Workspace)/image-builder/source-build-id/source-build-id.txt)"
-    displayName: Get Image Builder Tag
-  - script: >
-      docker build . -f ./eng/src/yaml-updater/Dockerfile -t yaml-updater
-    displayName: Build YAML Updater
-  - script: >
-      docker run --rm yaml-updater
-      ./eng/image-builder-tag-update-config.json
-      variables/imageNames.imageBuilderName
-      mcr.microsoft.com/dotnet-buildtools/image-builder:$(imageBuilderTag)
-      $(dotnetDockerBot.userName)
-      $(dotnetDockerBot.email)
-      $(BotAccount-dotnet-docker-bot-PAT)
-      dotnet
-      docker-tools
-      main
-    displayName: Run YAML Updater
+  - pwsh: |
+      Write-Host "##[section]Resolving ImageBuilder digest"
+      $repository = "mcr.microsoft.com/dotnet-buildtools/image-builder"
+      $image = "${repository}:latest"
+      $digest = (docker buildx imagetools inspect $image --format '{{.Manifest.Digest}}')
+
+      if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($digest)) {
+          Write-Host "##vso[task.logissue type=error]Unable to resolve a multi-platform digest for '$image'."
+          exit 1
+      }
+
+      $imageBuilderRef = "$repository@$($digest.Trim())"
+      Write-Host "$image refers to $imageBuilderRef"
+
+      Write-Host "##[section]Building ImageBuilder.Updater"
+      & pwsh ./eng/common/build.ps1 `
+          -restore `
+          -build `
+          -configuration Release `
+          -projects src/ImageBuilder.Updater/ImageBuilder.Updater.csproj
+
+      if ($LASTEXITCODE -ne 0) {
+          Write-Host "##vso[task.logissue type=error]Failed to build ImageBuilder.Updater."
+          exit 1
+      }
+
+      Write-Host "##[section]Running ImageBuilder.Updater"
+      & ./.dotnet/dotnet run `
+          --no-build `
+          --configuration Release `
+          --project src/ImageBuilder.Updater/ImageBuilder.Updater.csproj `
+          -- `
+          $imageBuilderRef
+
+      if ($LASTEXITCODE -ne 0) {
+          Write-Host "##vso[task.logissue type=error]ImageBuilder.Updater failed."
+          exit 1
+      }
+
+      Write-Host "ImageBuilder.Updater finished"
+    displayName: Run ImageBuilder Updater
+    env:
+      GITHUB_TOKEN: $(BotAccount-dotnet-docker-bot-PAT)
+      GITHUB_USER: $(dotnetDockerBot.userName)
+      GITHUB_EMAIL: $(dotnetDockerBot.email)