Skip to content

PP-QUANT-001 Q2 (gguf/cuda): retype the boundary — raw u32 qtype → GgmlType; ratchet 4 files → 0 #3438

Description

@noahgift

Parent: #3421 · Slot: 0.70.0, after M1+M2 (ratcheted: if it does not fit the train, the remainder rides the next tag with a slipped_from: note).
Mechanism: expert_swiglu_cuda.rs handles Q4_K/Q6_K/Q8_0, matmul.rs Q4_K/Q6_K, cuda.rs Q4_K only; 15 UnsupportedOperation sites across the four files. Device-behaviour claims in this PR need the CUDA-docs review arm.
Fix: parse u32 → GgmlType once at the loader boundary; everything downstream matches exhaustively on the enum. One PR for this directory only.
Falsifier: a non-increasing ratchet on files naming GGUF_TYPE_* or matching a bare integer qtype in this directory (baseline 4, measured on a672677f4); adding a new raw-u32 match → RED. Qwen2.5-Coder tok/s within APR-PERF-GATE-001 on the merge sha, same host.

Ĵ: 24 turns [A] — borrowed; estimate.sh aprender pools 0 of 47 ledger rows today, so there is no measured basis.
Plan + evidence: #3427 · design discussion: #3418

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priorityenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions