Skip to content

fix: point get-kernel-check at the built variant - #802

Open
drbh wants to merge 2 commits into
mainfrom
fix-get-kernel-check-variant
Open

fix: point get-kernel-check at the built variant#802
drbh wants to merge 2 commits into
mainfrom
fix-get-kernel-check-variant

Conversation

@drbh

@drbh drbh commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

this pr fixes the macos ci failures where get-kernel-check rejects a kernel that built fine. first hit on the einops port in huggingface/kernels-community#1092

tldr; the check handed $out to the resolver and let it pick a variant by detecting the backend at runtime. metal is the only backend detected from live hardware (torch.backends.mps.is_available(), everything else is baked into the torch build). the sandboxed macos build cannot see the gpu, so the torch-metal variant is never selected and the check fails even though the artifact is fine

repro (no kernel build needed)

mkdir -p /tmp/fake-kernel/torch-metal && touch /tmp/fake-kernel/torch-metal/metadata.json
kernels/.venv/bin/python -c "from pathlib import Path; import kernels; kernels.get_local_kernel(Path('/tmp/fake-kernel'))"
ValueError: Cannot parse metadata from `"/tmp/fake-kernel/torch-metal/metadata.json"`: EOF while parsing a value at line 1 column 0

on a mac with a visible gpu the variant is selected fine (it gets as far as reading the empty metadata.json). same command inside a seatbelt sandbox

sandbox-exec -p '(version 1)(allow default)(deny iokit-open)' \
    kernels/.venv/bin/python -c "from pathlib import Path; import kernels; kernels.get_local_kernel(Path('/tmp/fake-kernel'))"
FileNotFoundError: Cannot find a build variant for this system in /tmp/fake-kernel:

sandbox-exec is the same seatbelt mechanism the nix darwin sandbox uses. denying iokit hides the gpu, mps.is_available() returns false, and the same directory no longer resolves

note* this is also why only some metal builds fail in ci. arch metal builds set __noChroot = metalSupport since they need the host metal toolchain, so the gpu is still visible during the check. noarch metal builds like einops run fully sandboxed and always fail. the first command also fails on linux since there is no mps at all

the check now globs $out for */metadata.json and points the resolver at that exact variant dir instead of auto detecting from the build host. each extension derivation produces exactly one variant, so zero or multiple variants still fail the check

# The build host may not expose the accelerator being targeted (e.g. Metal is
# undetectable inside the sandboxed macOS build), so point the resolver at the
# derivation's single variant instead of relying on backend auto-detection.
[variant] = (p.parent for p in Path(out).glob("*/metadata.json"))

@danieldk danieldk Sep 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, I don't think we should rely on a wildcard. It weakens the test.

At any rate, what I think we should do:

  • Add a variant argument to the arch.nixs and no-arch.nix.
  • Expose variant through the envattribute.
  • Validate in this hook that variant is set and then use it.
  • Where the build derivations (arch/no-arch) is called, pass something like variant = buildSet.variants.kernelVariant kernelConfig;

Then we are not only testing the correct variant, but would also nicely get CI errors if the build is generating another variant than what is expected (which a wildcard would happily pass).

@danieldk danieldk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants