vulkbench - Vulkan Multi-Precision TOPS/GFLOPS Benchmark

A minimal C benchmark that measures int4, int6, int8, fp16, and fp32 compute throughput using Vulkan compute pipelines. Reports results in TOPS / TFLOPS.

What it does

• Creates Vulkan compute pipelines for 5 precisions: INT4 (packed), INT6, INT8, FP16, FP32
• 1 MB input buffer (power-of-2, fits L2 cache) for compute-bound measurement
• Uses Vulkan timestamp queries for accurate GPU-side measurement
• Runs 2 warmup + 3 measured iterations per precision
• VRAM allocation test with binary search for max allocatable buffer + bandwidth
• Prints a comparison summary table of best results per precision

Requirements

• Linux with Vulkan driver installed
• gcc, glslang-tools, spirv-as (spirv-tools), python3
• GPU with shaderInt8 support (Vulkan 1.2)
• NVIDIA Turing+ recommended for the INT8 DP4A path (VK_KHR_shader_integer_dot_product)

Build & Run

cd vulkbench
make          # build
make run      # build + run (compute benchmarks only)
make run-vram # build + run incl. VRAM alloc/BW/pressure tests
make clean    # remove artifacts

./vulkbench [-vram] [-n X] [-h]
# -vram: run VRAM allocation/BW/pressure tests (default: compute only)
# -n X:  dispatches per submission, default 50 (compiled-in default via make BATCH=N)
# -h:    usage

Example Output (RTX 2080 Ti, ./vulkbench, -n 50 default)

=== Device Info ===
  Name:          NVIDIA GeForce RTX 2080 Ti
  Type:          Discrete GPU
  Driver:        550.652.64
  API version:   1.3.277
  shaderInt8:    yes
  shaderFloat16: yes
  intDotProduct: yes (DP4A path available)
  Dispatch:      1048576 threads (4096 x 256)
  Batch:         50 dispatches per submission
  INT4/6/8 kernel: DP4A (OpSDot, 32 dots/iter)

=== Compute-Bound Benchmark ===
  --- INT4 ---
    Run  1:  655.333 ms  ->   122.88 TOPS
    Run  2:  662.979 ms  ->   121.47 TOPS
    Run  3:  658.706 ms  ->   122.26 TOPS
    Best: 122.88  Avg: 122.20
  --- INT6 ---
    Run  1:  674.946 ms  ->    59.66 TOPS
    Run  2:  672.651 ms  ->    59.86 TOPS
    Run  3:  669.266 ms  ->    60.16 TOPS
    Best: 60.16  Avg: 59.89
  --- INT8 ---
    Run  1:  685.450 ms  ->    58.74 TOPS
    Run  2:  688.556 ms  ->    58.48 TOPS
    Run  3:  689.343 ms  ->    58.41 TOPS
    Best: 58.74  Avg: 58.54
  --- FP16 ---
    Run  1: 1632.903 ms  ->    24.66 TFLOPS
    Run  2: 1630.592 ms  ->    24.69 TFLOPS
    Run  3: 1634.654 ms  ->    24.63 TFLOPS
    Best: 24.69  Avg: 24.66
  --- FP32 ---
    Run  1: 2483.148 ms  ->    16.22 TFLOPS
    Run  2: 2485.123 ms  ->    16.20 TFLOPS
    Run  3: 2485.139 ms  ->    16.20 TFLOPS
    Best: 16.22  Avg: 16.21

=========================================
  VulkBench Results
=========================================
  INT4     122.88 TOPS  (best)
  INT6      60.16 TOPS  (best)
  INT8      58.74 TOPS  (best)
  FP16      24.69 TFLOPS  (best)
  FP32      16.22 TFLOPS  (best)
=========================================

Tuning

Edit the #define values at the top of main.c:

Parameter	Default	Description
WORKGROUP_SIZE	256	Threads per workgroup
NUM_WORKGROUPS	4096	Number of dispatched workgroups
BATCH_DISPATCHES	50	Dispatches/submission: ./vulkbench -n X
INPUT_BUF_MB	1	Input buffer size in MB (L2 cache fit)
WARMUP_ITERS	2	Warmup dispatches (not measured)
BENCH_ITERS	3	Measured iterations

Expected Results by Architecture

The INT4/INT6/INT8 kernels simulate lower-precision arithmetic by holding values in int16 lanes (4/6/8-bit multiplier ranges). They all pack into the same v_pk_mad_u16 slots, so on GFX9 every kernel issues at the identical ~3.0-3.1 TOPS instruction rate. The INT4 result is reported at 2x the op count (OPS_PER_THREAD_INT4 = 2 × the other precisions) because a 4-bit value packs 2 per 8-bit slot — i.e. an int4 GEMM does 2x the element-MACs of an int8 GEMM in the same hardware budget. This mirrors the 2x INT4:INT8 ratio delivered by native packed-int4 hardware (RDNA3 DPX).

Architecture	INT4 vs INT8	Reason
GFX9 (Vega/Renoir)	2.0× (accounted)	No native sub-int8 MAC; INT4 op count is 2x INT8 (packing density)
RDNA3+	~2× (native)	Native packed int4 DPX instructions

The INT4/INT6/INT8 layout keeps 8 i16vec2 accumulators (c0..c7), each fed 4 distinct products per iteration from 4 snapshot vectors (a0..a3 = copies of c0..c3). The a-values are full-register copies (coalesced to ~4 moves), so ACO keeps every loop-carried value packed in one VGPR and emits 32 pure v_pk_mad_u16 per iteration — no lane scalarization. INT4/INT6 use the same structure with 4-bit (≤ 15) and 6-bit (≤ 63) multiplier ranges; all low-bit formats pack into the same native v_pk_mad_u16 slots on GFX9 (no sub-int8 MAC).

INT4/INT6/INT8 DP4A Kernels (NVIDIA)

shader_int*_dp4a.spvasm are hand-written SPIR-V kernels using packed OpSDotKHR (SPV_KHR_integer_dot_product, DP4A datapath). All three share one issue stream — 8 int32 accumulators × 4 snapshot packs × 32 dots/iter × 3000 iters — so wall times are identical; only the B-operand byte ranges differ (≤15 / ≤63 / ≤127) and the op accounting:

main.c queries shaderIntegerDotProduct, enables VK_KHR_shader_integer_dot_product on the device, and auto-selects the DP4A trio when supported (else the generic i16vec2 trio). Override for A/B testing: VULKBENCH_DP4A=all|none (legacy VULKBENCH_INT8=generic|dp4a still honored). Needed because glslang 15.2 lacks GL_EXT_shader_integer_dot_product; assembled with spirv-as (validated: spirv-val --target-env vulkan1.2 clean, 32 OpSDot each).

Kernel	Ops/thread	Reported vs INT8	Reason
INT4-DP4A	1,536,000	~2×	Packing-density convention: 2 int4 values per 8-bit slot (same convention as the GFX9 v_pk_mad_u16 era)
INT6-DP4A	768,000	~1×	No native 6-bit MAC — values ride full byte lanes, identical issue stream to INT8
INT8-DP4A	768,000	1×	Baseline DP4A rate

Measured on RTX 2080 Ti ( -n 50 default; generic column via VULKBENCH_DP4A=none at the same settings):

Kernel	DP4A (best)	Generic (best)	Speedup
INT4	122.88 TOPS	32.65 TOPS	3.8×
INT6	60.16 TOPS	15.81 TOPS	3.8×
INT8	58.74 TOPS	15.76 TOPS	3.7×

Wall times are identical across the trio (~13 ms per dispatch at boost), so the ordering is purely the accounting convention above. (Generic INT4 was already 2× generic INT8/INT6 for the same reason.)

Remaining headroom to vendor tensor specs needs cooperative-matrix WMMA tensor cores (VK_KHR_cooperative_matrix / VK_NV_cooperative_matrix, both advertised on this card) — planned follow-up; DP4A is the CUDA-core ceiling (~4× generic).

VRAM Bandwidth Columns

Per-row Wr/Rd GB/s measures CPU memset/memcpy on the mapped BAR — on ReBAR systems this is BAR/CPU bound and does not track GDDR clocks. GPUwr/GPUrd use copy engines between a 64 MB device-only pair and staging with GPU timestamps (plus an untimed warmup copy), so they track the real VRAM fabric — verify with a VRAM-freq sweep (expect GPU columns to scale, CPU columns to stay flat).

Heap selection: pressure tests use the largest DEVICE_LOCAL heap. On dGPUs without ReBAR the preferred host-visible type is a small (~246 MB) BAR window, so the test switches to the device-only GDDR heap with staging (VRAM test heap: log line); otherwise sizes cap at the BAR and nothing measured touches GDDR.

RTX 2080 Ti reference (./vulkbench -vram, no ReBAR): single-buffer cap 11008 MB, max allocatable 10964 MB ≈ full 11 GB GDDR

FP16 Dot-Product Shader

shader_fp16.comp is a 4x4 GEMM with all-f16 accumulators. Each iteration issues 16 f16vec4 dot() calls which ACO lowers to native v_dot4_f32_f16 (4 packed FP16 multiplies + 1 FP32 add per issue). Everything stays packed f16 — no f32↔f16 conversions in the loop — and a ping-pong dependency (a = f16vec4(c)) prevents the dots from being hoisted or eliminated.

Ops/iter: 16 dots × (4 mul + 4 add) = 128; 6000 iterations = 768,000 ops/thread (same count as the other precisions).

Measured in isolation (spvtest, 4096 workgroups):

• FP32 baseline (FMA): 1.790 TOPS
• FP16 dot GEMM: 1.821 TOPS (stable +1.7% across interleaved A/B)

The dot instruction executes ~1 per 4 cycles per lane, so this stays near the machine's plain-FMA ceiling rather than reaching the theoretical packed-FP16 2× rate. In the full multi-precision suite the longer sustained load brings all precisions to the same ~1.75 ceiling, so the advantage is visible mainly in per-dispatch isolation.

Files

File	Purpose
main.c	Vulkan init, pipeline creation, benchmark loop, VRAM test
shader_int4.comp	GLSL compute shader: packed int16 GEMM kernel (4-bit multipliers, 8x unrolled, v_pk_mad_u16)
shader_int6.comp	GLSL compute shader: packed int16 GEMM kernel (6-bit multipliers, 8x unrolled, v_pk_mad_u16)
shader_int8.comp	GLSL compute shader: packed int16 GEMM kernel (int8 in int16 lanes, 8x unrolled, v_pk_mad_u16)
shader_fp16.comp	GLSL compute shader: fp16 4x4 GEMM using f16vec4 dot() (native v_dot4_f32_f16 on GFX9)
shader_fp32.comp	GLSL compute shader: fp32 MAC kernel
shader_int8_dp4a.spvasm	Hand-written SPIR-V asm: INT8 DP4A kernel (packed OpSDot, see below)
shader_int4_dp4a.spvasm	Hand-written SPIR-V asm: INT4 DP4A kernel (4-bit B packs, 2× density count)
shader_int6_dp4a.spvasm	Hand-written SPIR-V asm: INT6 DP4A kernel (6-bit B packs, same rate as INT8)
spv2c.py	Converts SPIR-V binaries to C arrays
Makefile	Build system (spirv-as rule for .spvasm, BATCH stamp tracking)