Skip to content

[Matrix] Add comparison operator tests - #1447

Merged
farzonl merged 8 commits into
llvm:mainfrom
farzonl:task/issue-1446
Sep 16, 2026
Merged

farzonl merged 8 commits into
llvm:mainfrom
farzonl:task/issue-1446

Conversation

@farzonl

@farzonl farzonl commented Aug 18, 2026

Copy link
Copy Markdown
Member

resolves #1446

cover cmp operators across matrix sizes and base types

assisted by co-pilot MAI model

@farzonl farzonl added the test-all When applied to a PR this will opt-in to additional pre-merge test configurations.. label Aug 18, 2026
@farzonl farzonl changed the title Task/issue 1446 [Matrix] Add comparison operator tests Aug 18, 2026
@farzonl

farzonl commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

So I don't have to come back and remove a bunch of xfails lets wait on this PR until llvm/llvm-project#216791 merges

@farzonl

farzonl commented Aug 19, 2026

Copy link
Copy Markdown
Member Author

the tests that have 64/16 bit requires are not as exhaustive with operator testing as the 32bit tests.

@kmpeng
kmpeng self-requested a review August 19, 2026 17:22
@farzonl

farzonl commented Aug 20, 2026 •

Copy link
Copy Markdown
Member Author

Failures to investigate:

DXC Metal

FAIL: OffloadTest-mtl :: Basic/Matrix/matrix_comparison_operators.uint32.test (5 of 621)
******************** TEST 'OffloadTest-mtl :: Basic/Matrix/matrix_comparison_operators.uint32.test' FAILED ********************
Exit Code: -6

Command Output (stdout):
--
# RUN: at line 85
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/split-file /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/test/Basic/Matrix/matrix_comparison_operators.uint32.test /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/split-file /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/test/Basic/Matrix/matrix_comparison_operators.uint32.test /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp
# RUN: at line 86
"/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/DXC/build/bin/dxc-3.7" -T cs_6_0 -Fo /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/source.hlsl
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/DXC/build/bin/dxc-3.7 -T cs_6_0 -Fo /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/source.hlsl
# RUN: at line 87
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# .---command stdout------------
# | Using Metal API
# | Descriptor heap created with 2 descriptors.
# | Creating UAV: { Size = 48, Register = u0, Space = 0, HasCounter = 0 }
# `-----------------------------
# .---command stderr------------
# | Validation Warning: [ mvk-warn ]
# | VK_ERROR_FEATURE_NOT_PRESENT: Metal does not support buffer robustness.
# | 
# | Unsupported Resource format specified
# | UNREACHABLE executed at /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/lib/API/MTL/MTLDevice.cpp:128!
# | PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace and instructions to reproduce the bug.
# | Stack dump:
# | 0.	Program arguments: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# | Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
# | 0  offloader                0x0000000102b5d208 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
# | 1  offloader                0x0000000102b5af50 llvm::sys::RunSignalHandlers() + 204
# | 2  offloader                0x0000000102b5dd08 SignalHandler(int, __siginfo*, void*) + 320
# | 3  libsystem_platform.dylib 0x000000018a843744 _sigtramp + 56
# | 4  libsystem_pthread.dylib  0x000000018a839888 pthread_kill + 296
# | 5  libsystem_c.dylib        0x000000018a73e850 abort + 124
# | 6  offloader                0x0000000102b26318 llvm::install_out_of_memory_new_handler() + 0
# | 7  offloader                0x0000000102ba483c (anonymous namespace)::MTLDevice::createResource(offloadtest::Resource&, unsigned long) + 912
# | 8  offloader                0x0000000102b9d500 (anonymous namespace)::MTLDevice::createBuffers(offloadtest::Pipeline&, (anonymous namespace)::MTLDevice::InvocationState&) + 928
# | 9  offloader                0x0000000102b967e0 (anonymous namespace)::MTLDevice::executeProgram(offloadtest::Pipeline&) + 1080
# | 10 offloader                0x0000000102b00998 main + 2072
# | 11 dyld                     0x000000018a471d54 start + 7184
# `-----------------------------
# error: command failed with exit status: -6

Clang DXC Metal

******************** TEST 'OffloadTest-clang-mtl :: Basic/Matrix/matrix_comparison_operators.uint32.test' FAILED ********************
Exit Code: -6

Command Output (stdout):
--
# RUN: at line 85
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/split-file /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/test/Basic/Matrix/matrix_comparison_operators.uint32.test /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/split-file /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/test/Basic/Matrix/matrix_comparison_operators.uint32.test /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp
# RUN: at line 86
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/clang-dxc --dxv-path=/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/DXC/build/bin -T cs_6_0 -Fo /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/source.hlsl
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/clang-dxc --dxv-path=/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/DXC/build/bin -T cs_6_0 -Fo /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/source.hlsl
# .---command stdout------------
# | Signed DxilContainer written to "/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o"
# | Validation succeeded.
# `-----------------------------
# RUN: at line 87
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# .---command stdout------------
# | Using Metal API
# | Descriptor heap created with 2 descriptors.
# | Creating UAV: { Size = 48, Register = u0, Space = 0, HasCounter = 0 }
# `-----------------------------
# .---command stderr------------
# | Validation Warning: [ mvk-warn ]
# | VK_ERROR_FEATURE_NOT_PRESENT: Metal does not support buffer robustness.
# | 
# | Unsupported Resource format specified
# | UNREACHABLE executed at /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/lib/API/MTL/MTLDevice.cpp:128!
# | PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace and instructions to reproduce the bug.
# | Stack dump:
# | 0.	Program arguments: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# | Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
# | 0  offloader                0x00000001007ed208 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
# | 1  offloader                0x00000001007eaf50 llvm::sys::RunSignalHandlers() + 204
# | 2  offloader                0x00000001007edd08 SignalHandler(int, __siginfo*, void*) + 320
# | 3  libsystem_platform.dylib 0x000000018a843744 _sigtramp + 56
# | 4  libsystem_pthread.dylib  0x000000018a839888 pthread_kill + 296
# | 5  libsystem_c.dylib        0x000000018a73e850 abort + 124
# | 6  offloader                0x00000001007b6318 llvm::install_out_of_memory_new_handler() + 0
# | 7  offloader                0x000000010083483c (anonymous namespace)::MTLDevice::createResource(offloadtest::Resource&, unsigned long) + 912
# | 8  offloader                0x000000010082d500 (anonymous namespace)::MTLDevice::createBuffers(offloadtest::Pipeline&, (anonymous namespace)::MTLDevice::InvocationState&) + 928
# | 9  offloader                0x00000001008267e0 (anonymous namespace)::MTLDevice::executeProgram(offloadtest::Pipeline&) + 1080
# | 10 offloader                0x0000000100790998 main + 2072
# | 11 dyld                     0x000000018a471d54 start + 7184
# `-----------------------------
# error: command failed with exit status: -6

The issue in Metal seems to be offloader specific. We don't have a uint32 Format.
lib/API/MTL/MTLDevice.cpp:128

static MTL::PixelFormat getMTLFormat(DataFormat Format, int Channels) {
  switch (Format) {
  case DataFormat::Int32:
    MTLFormats(32Sint) break;
  case DataFormat::Float32:
    MTLFormats(32Float) break;
  case DataFormat::UInt64:
  case DataFormat::Int64:
    if (Channels == 1)
      return MTL::PixelFormatRG32Uint;
    if (Channels == 2)
      return MTL::PixelFormatRGBA32Uint;
    llvm_unreachable("Unsupported channel count for 64-bit format");

  default:
    llvm_unreachable("Unsupported Resource format specified");
  }
  return MTL::PixelFormatInvalid;
}

@farzonl farzonl removed the test-all When applied to a PR this will opt-in to additional pre-merge test configurations.. label Aug 20, 2026
@farzonl
farzonl force-pushed the task/issue-1446 branch 2 times, most recently from 61d2f34 to b054c3a Compare August 20, 2026 17:40
Comment thread test/Basic/Matrix/matrix_comparison_operators.fp16.test Outdated
@damyanp
damyanp requested review from Icohedron and alsepkow August 25, 2026 17:23

@Icohedron Icohedron left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why the 32-bit tests have to be the only comprehensive ones.
The 32-bit tests aren't even that much bigger than the other tests in line count either.

e.g. the fp16 test is 72 lines while the fp32 test is 105 lines

Comment thread test/Basic/Matrix/matrix_comparison_operators.fp16.test Outdated
Comment thread test/Basic/Matrix/matrix_comparison_operators.fp32.test
Comment thread test/Basic/Matrix/matrix_comparison_operators.int64.test Outdated
Buffers:
- Name: UintIn
Format: UInt32
Data: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 ]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION] The int and uint tests should exercise values where the sign bit is actually set. As-is, there is no observable difference if the compiler used signed/unsigned operations where it should have been unsigned/signed.

@farzonl farzonl added the test-lavapipe Run lavapipe tests, only label Aug 26, 2026

@alsepkow alsepkow left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming @Icohedron comments are addressed.
And looks like there's a legalization issue causing a crash in the 32 bit tests?

@farzonl

farzonl commented Aug 27, 2026

Copy link
Copy Markdown
Member Author

LGTM assuming @Icohedron comments are addressed. And looks like there's a legalization issue causing a crash in the 32 bit tests?

Yes I am waiting on reviews for this pr that will fix that: llvm/llvm-project#219017 care to take a look? It is really short.

@Icohedron Icohedron left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to have uint tests exercise extra large integers which have the most significant bit set to be sure it's not interpreted as a negative signed integer instead.

But otherwise LGTM

@farzonl farzonl removed the test-lavapipe Run lavapipe tests, only label Sep 16, 2026
@farzonl
farzonl merged commit 8db118a into llvm:main Sep 16, 2026
28 checks passed
@farzonl
farzonl deleted the task/issue-1446 branch September 16, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Matrix] Add elementwise comparison operator tests

5 participants