Repository navigation
SRE-1172: Link a cached prebuilt AWS-LC in CI instead of compiling it every run - #10010
Merged
Merged
Conversation
`install-tools` now restores an AWS-LC install prefix from the Actions cache, building it from the workspace's `aws-lc-sys` on a miss, and points `aws-lc-sys` at it through `AWS_LC_SYS_SYSTEM_DIR_<host>` and `AWS_LC_SYS_STATIC_<host>`. The prefix holds the library, headers and bindings of an ordinary `dev` source build, so the linked result matches compiling it. Bench and CodSpeed jobs build with optimized profiles and keep compiling AWS-LC.
…thout the prebuilt AWS-LC
github-actions
Bot
added
the
area/infra
![semgrep-code-hashintel[bot]](https://avatars.githubusercontent.com/u/22116112?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #10010 +/- ##
==========================================
- Coverage 67.97% 67.96% -0.02%
==========================================
Files 1945 1946 +1
Lines 201707 201826 +119
Branches 8288 8314 +26
==========================================
+ Hits 137107 137163 +56
- Misses 62981 63039 +58
- Partials 1619 1624 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will not alter performance
|
…the environment Every job that installs Rust now links a prebuilt AWS-LC. Jobs building with an optimized profile select a `release` prefix instead of compiling AWS-LC, and each profile has its own cache entry. The composite action reads expressions through `env` rather than interpolating them into scripts.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
`build-prefix.sh` now downloads the locked `aws-lc-sys` crate, checks it against the Cargo.lock checksum, and builds and installs its bundled AWS-LC with CMake and the Rust bindings that build generates, keeping the crate's symbol prefix. The cache key moves to `cache-key.py`, which reads Cargo.lock with `tomllib`.
The CI mise environment pins `bindgen-cli`, and the cache key includes the version on `PATH`. `build-prefix.sh` reports its build time as a notice, which marks the jobs that missed the cache.
`bindgen-cli` moves from the `ci` mise environment to an `aws-lc` environment that `build-prefix.sh` installs on a cache miss, so other jobs no longer install it. The cache key covers the pinned version through the environment's config and lockfile, and a failing compiler version probe names the command. The temporary compile timings workflow is removed.
indietyp
previously approved these changes
Oct 13, 2026
indietyp
requested review from
TimDiekmann
and
a balanced review from Copilot
October 13, 2026 07:24
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The cache keying, profile wiring, native build configuration, and environment propagation are internally consistent.
0 open findings
What changed in this PR
Adds cached, prebuilt AWS-LC artifacts to reduce Rust compilation time in CI.
Changes:
- Adds an action to build, cache, restore, and link AWS-LC.
- Configures benchmark jobs to use release artifacts.
- Installs pinned
bindgen-cliand forwards AWS-LC environment variables through Turbo.
| File | Description |
|---|---|
turbo.json |
Passes AWS-LC variables to Turbo tasks. |
.github/workflows/codspeed.yml |
Selects release AWS-LC for CodSpeed. |
.github/workflows/bench.yml |
Selects release AWS-LC for benchmarks. |
.github/actions/install-tools/action.yml |
Integrates AWS-LC installation into Rust setup. |
.github/actions/install-aws-lc/build-prefix.sh |
Builds and installs prefixed AWS-LC artifacts. |
.github/actions/install-aws-lc/action.yml |
Implements cache restoration, creation, and environment setup. |
.config/mise/mise.ci.lock |
Locks the new bindgen tool version. |
.config/mise/config.ci.toml |
Adds bindgen-cli to CI tooling. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
TimDiekmann
reviewed
Oct 13, 2026
TimDiekmann
left a comment
TimDiekmann
left a comment
Member
There was a problem hiding this comment.
Review written by Claude on behalf of Tim.
Much smaller now, thanks. One thing on the cache key inline, the rest are nits:
install-tools/action.yml:21: theaws_lc_profiledescription is hard to parse. I'd reuse the wording ofprofileininstall-aws-lc.- The PR body says "once per
aws-lc-sysversion, profile and toolchain", but the key has no toolchain in it (and doesn't need one). build-prefix.sh:21:dirname "$(…)"swallows a failingcargo metadata, so the error then surfaces as cmake not finding./aws-lc.action.yml:22: bench installs base and head into the same$RUNNER_TEMP/aws-lc-release, so a miss on head installs on top of the base prefix. Anrm -rf "$prefix"before configuring keeps the cached tree clean.
The key uses `ImageOS` instead of `RUNNER_OS`, so jobs on `ubuntu-latest` stop sharing an install with `ubuntu-24.04` jobs once the label moves to a newer image. `build-prefix.sh` fails with its own error when `cargo metadata` fails, removes a previous install before building, and names the image in its notice. The `aws_lc_profile` description reuses the `profile` wording.
indietyp
reviewed
Oct 13, 2026
TimDiekmann
approved these changes
Oct 13, 2026
indietyp
approved these changes
Oct 13, 2026
Contributor
Author
|
@TimDiekmann the four nits from your review are in 14a0490:
🤖 Generated with Claude Code |
indietyp
enabled auto-merge
October 13, 2026 09:36
Contributor
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1002 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1527 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 108 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 27 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |
claude
Bot
requested review from
TimDiekmann and
indietyp
and removed request for
TimDiekmann and
indietyp
October 13, 2026 09:58
This was referenced Oct 13, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested via Slack thread
🌟 Purpose
Every CI job that compiles Rust rebuilds AWS-LC from C in the
aws-lc-sysbuild script. sccache already serves those 391 C and assembly objects, but each hit is a fetch from R2 (about 0.17s) plus a local preprocessor run. sccache's direct mode, which skips that run, only works with a local disk cache. The build script also gets one jobserver token while the rest of the graph is compiling, so the fetches run almost one at a time. In thehash-graphcompile that is about 100 of 290 seconds (#10005), andrustls,reqwest,tonicand everything above them wait for it.This PR builds AWS-LC once per
aws-lc-sysversion, runner image and architecture, and profile, stores the install in the Actions cache, and hasaws-lc-syslink it instead of compiling AWS-LC.🔍 Changes
install-toolsruns the newinstall-aws-lcaction in every job that installs Rust. The action restores the AWS-LC install from the Actions cache, building it on a miss. It then setsAWS_LC_SYS_SYSTEM_DIR_<host>andAWS_LC_SYS_STATIC_<host>=1, which putaws-lc-sys0.45.0 on its system-library path.build-prefix.shtakes theaws-lc-syssource cargo fetched and checksummed forcargo metadata, builds the AWS-LC it bundles with AWS-LC's own CMake build, and installs the static library, headers and the Rust bindings that build generates withGENERATE_RUST_BINDINGS. The build is static and crypto-only, needs no Go or Perl because the crate ships pregenerated assembly, and keeps the crate'saws_lc_0_45_0symbol prefix.bindgen-cliis a CI mise tool.dev, and the bench and CodSpeed jobs, which build with the optimizedbenchprofile, passaws_lc_profile: releaseand link aReleasebuild.aws-lc-sysversion fromcargo pkgid, the runner image (ImageOS) and architecture, the profile, and a hash of the action's files and the CI mise config and lockfile, which pinbindgen-cli.cargo pkgidreads Cargo.lock without downloading anything. There are no restore keys, so any change builds a new install.bindgen-cli.turbo.jsonpassesAWS_LC_SYS_*through, so tasks in strict environment mode see the variables.hash-graphcompile in CI,cargo build --bin hash-graph --all-featuresonubuntu-24.04, measured with a temporary workflow that this PR has since removed:aws-lc-sysbuild scriptLocally, built against the install, the
hash-graph-authentication(135) andhash-secret(48) unit tests pass, andhash-graphlinks. A rustls TLS 1.3 handshake plus AES-GCM, Ed25519, X25519 and HKDF checks give the same results against thedevandreleaseinstalls as against a source build. The symbols the bindings name are all defined in the installed library, apart from the FIPS-onlyBORINGSSL_integrity_test.install-toolsor access to the Actions cache, so they still build AWS-LC from source.🤖 Generated with Claude Code
https://claude.ai/code/session_01MY9Fuaaeq21mwQc4hdjxYM