Skip to content

Migrate release CDN URLs to repo.amd.com stream layout - #85

Merged
HereThereBeDragons merged 3 commits into
developfrom
users/lpromber/repo_amd_com_cdn_urls
Aug 25, 2026
Merged

Migrate release CDN URLs to repo.amd.com stream layout#85
HereThereBeDragons merged 3 commits into
developfrom
users/lpromber/repo_amd_com_cdn_urls

Conversation

@HereThereBeDragons

Copy link
Copy Markdown
Collaborator

Summary

Update the release CDN URL derivation to the new repo.amd.com stream layout
introduced by RFC0012 and documented in ROCm/TheRock#7553. Each release stream
now serves from its own <stream>.repo.amd.com/rocm/ subdomain, replacing the
former rocm.<channel>.amd.com hosts and *-multi-arch/ paths.

Only the two streams the status updater handles are remapped (nightly and
prerelease); stable, bkc, and dev are out of scope here since the updater
does not process them.

Details

URL mapping applied in derive_release_cdn_urls:

Artifact Old New
base (nightly) https://rocm.nightlies.amd.com/ https://nightly.repo.amd.com/rocm/
base (prerelease) https://rocm.prereleases.amd.com/ https://rc.repo.amd.com/rocm/
tarball tarball-multi-arch/ core/tarball/
wheels whl-multi-arch/ whl-next/
packages packages-multi-arch/ core/packages/

The dated <YYYYMMDD>-<run_id> segment for nightly native deb/rpm packages is
preserved, so a nightly deb index resolves to e.g.
https://nightly.repo.amd.com/rocm/core/packages/deb/20260824-32675368657/.

Test plan

  • pytest for therock_classify, therock_update_status_json, and
    therock_status_document (257 passed)

@HereThereBeDragons
HereThereBeDragons requested review from a team and marbre August 24, 2026 14:38
tarball_url=f"{base}tarball-multi-arch/",
wheels_url=f"{base}whl-multi-arch/",
tarball_url=f"{base}core/tarball/",
wheels_url=f"{base}whl-next/",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the aggregated index for all our packages. Is that desired here or do you want to use the per-product local indexes, e.g. {nightly,dev}.repo.amd.com/rocm/{core,pytorch,jax}/whl-next? Those are not self-consistent but depends on what you want and need.

@marbre marbre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rpm and deb are the underlying folders in S3 but consumers are rather guided to use OS-profiles. What is this captured for and what is the intended use?

@HereThereBeDragons

Copy link
Copy Markdown
Collaborator Author

to both of your comments:
the urls should be used for automated ingestion of the wheels/packages for the downstream pipeline. while the release ci is still running the urls will be the one pointing to the s3 artifact bucket. after publish to release bucket successfully ran we update to the cdn urls. e.g.

      "urls": {
        "tarballs": "https://rocm.nightlies.amd.com/tarball-multi-arch/",
        "wheels": "https://rocm.nightlies.amd.com/whl-multi-arch/",
        "artifacts": "https://therock-nightly-artifacts.s3.amazonaws.com/32675368657-linux/index.html",
        "rpm": "https://rocm.nightlies.amd.com/packages-multi-arch/rpm/20260824-32675368657/",
        "deb": "https://rocm.nightlies.amd.com/packages-multi-arch/deb/20260824-32675368657/"
      },

from the status.json you get "rocm_version": "10.1.0a20260824", and the urls. that should be enough to get the rocm installation parameterized and automatized for your downstream build/test process.

that is why i also used the top-level whl-next/ and not the product local ones.

@marbre

marbre commented Aug 24, 2026

Copy link
Copy Markdown
Member

to both of your comments: the urls should be used for automated ingestion of the wheels/packages for the downstream pipeline. while the release ci is still running the urls will be the one pointing to the s3 artifact bucket. after publish to release bucket successfully ran we update to the cdn urls. e.g.

      "urls": {
        "tarballs": "https://rocm.nightlies.amd.com/tarball-multi-arch/",
        "wheels": "https://rocm.nightlies.amd.com/whl-multi-arch/",
        "artifacts": "https://therock-nightly-artifacts.s3.amazonaws.com/32675368657-linux/index.html",
        "rpm": "https://rocm.nightlies.amd.com/packages-multi-arch/rpm/20260824-32675368657/",
        "deb": "https://rocm.nightlies.amd.com/packages-multi-arch/deb/20260824-32675368657/"
      },

from the status.json you get "rocm_version": "10.1.0a20260824", and the urls. that should be enough to get the rocm installation parameterized and automatized for your downstream build/test process.

that is why i also used the top-level whl-next/ and not the product local ones.

If this is for downstream consumers you may rather want to point them to a generic https://rocm.nightlies.amd.com/packages-multi-arch/<os-profile>/20260824-32675368657/ and than it is up to them to use the correct profile. Please try to get a comment from Nirmal on this.

Using the aggregated Python index however is what you want.

@nunnikri

nunnikri commented Aug 24, 2026

Copy link
Copy Markdown

to both of your comments: the urls should be used for automated ingestion of the wheels/packages for the downstream pipeline. while the release ci is still running the urls will be the one pointing to the s3 artifact bucket. after publish to release bucket successfully ran we update to the cdn urls. e.g.

      "urls": {
        "tarballs": "https://rocm.nightlies.amd.com/tarball-multi-arch/",
        "wheels": "https://rocm.nightlies.amd.com/whl-multi-arch/",
        "artifacts": "https://therock-nightly-artifacts.s3.amazonaws.com/32675368657-linux/index.html",
        "rpm": "https://rocm.nightlies.amd.com/packages-multi-arch/rpm/20260824-32675368657/",
        "deb": "https://rocm.nightlies.amd.com/packages-multi-arch/deb/20260824-32675368657/"
      },

from the status.json you get "rocm_version": "10.1.0a20260824", and the urls. that should be enough to get the rocm installation parameterized and automatized for your downstream build/test process.
that is why i also used the top-level whl-next/ and not the product local ones.

If this is for downstream consumers you may rather want to point them to a generic https://rocm.nightlies.amd.com/packages-multi-arch/<os-profile>/20260824-32675368657/ and than it is up to them to use the correct profile. Please try to get a comment from Nirmal on this.

Using the aggregated Python index however is what you want.

This is going to be tricky. Either we should give a generic url as suggested by Marius.
https://rocm.nightlies.amd.com/packages-multi-arch/<os-profile>/20260824-32675368657/

Or we should select a default OS profile. and give the other options available. If this is possible in quartz.

OS_PROFILE=ubuntu2404 # Select your from the available options. debian12, debian13, ubuntu2204, ubuntu2604, rhel8, rhel9, rhel10, sles15, sles16, azl3
https://rocm.nightlies.amd.com/packages-multi-arch/$OS_PROFILE/20260824-32675368657/

@marbre

marbre commented Aug 24, 2026

Copy link
Copy Markdown
Member

Technically rocm/core/packages/{deb,rpm}/ does work for the pure file location (maybe not well for signed package repositories?) but I would then suggest to point to rocm/core/packages/<os-profile>/.

@HereThereBeDragons

Copy link
Copy Markdown
Collaborator Author

as you see we have above

        "rpm": "https://rocm.nightlies.amd.com/packages-multi-arch/rpm/20260824-32675368657/",
        "deb": "https://rocm.nightlies.amd.com/packages-multi-arch/deb/20260824-32675368657/"

i thought having a fully working url for copy&paste would be nicer. if rpm/deb are not preferred, we could maybe go for the preferred default OS for each of them? i guess ubuntu24 and rhel10? or the <os_profile>.

otherwise:
we also have the other possibility to use rpm/deb here to have working links, but then in our little "consumer api" can offer to rewrite the link to the specific OS profile? (https://github.com/ROCm/Quartz/blob/main/scripts/consumer/read_status_json.py , functionality is not there yet)

@marbre

marbre commented Aug 24, 2026

Copy link
Copy Markdown
Member

Don't make it too fancy :)

@HereThereBeDragons

Copy link
Copy Markdown
Collaborator Author

ok i will stick with rpm/deb as it is right now and we will see if at a later point any of the consumers want something different.

@marbre

marbre commented Aug 25, 2026

Copy link
Copy Markdown
Member

ok i will stick with rpm/deb as it is right now and we will see if at a later point any of the consumers want something different.

As discussed above, consumers should rather use <os-profile>. Not making is fancy rather means just pass the placeholder and let them figure out the profile they want to use instead of providing rewrite links to what a customer might desire.

@HereThereBeDragons
HereThereBeDragons merged commit 8f9066b into develop Aug 25, 2026
2 checks passed
@HereThereBeDragons
HereThereBeDragons deleted the users/lpromber/repo_amd_com_cdn_urls branch August 25, 2026 19:14
quartz-sync-github-app Bot pushed a commit that referenced this pull request Aug 25, 2026
8f9066b, Migrate release CDN URLs to repo.amd.com stream layout (#85), Laura Promberger (laura.promberger@amd.com), Tue Aug 25 21:14:28 2026 +0200
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants