QVAC-25039 fix: stop pinning gpu_layers so the qvac-fabric fit can run

[iancris]

🎯 Problem

LLM_CONFIG_DEFAULTS injected gpu_layers: 99 into every LLM load. Fabric reads any set value as user intent and refuses to override it, so its automatic GPU/CPU placement aborted before it could pick anything:

common_fit_params: failed to fit params to free device memory: n_gpu_layers already set by user to 99, abort

The best the SDK config could express was plain layer offload, and a model larger than VRAM was offloaded whole and spilled back to host memory by the driver — 2.5 tok/s on the reporter's box against 22 tok/s for the placement the fit would have chosen. This is already noted in model-fit/calibration/METHODOLOGY.md: "The SDK pins n_gpu_layers: 99, so llama.cpp's own fit cannot reduce the..."

This is one of three independent causes on QVAC-25039. The other two are fixed in their own PRs (the sharded load path in inference-addon-cpp, llm-llamacpp's override buffer); none of the three depends on another.

📝 How

The default is dropped, and nothing is substituted for it.

With gpu_layers unset the addon receives fabric's own -1 sentinel, and llama already resolves a negative n_gpu_layers to every layer:

// src/llama-model.cpp
uint32_t llama_model::n_gpu_layers() const {
    return params.n_gpu_layers >= 0 ? params.n_gpu_layers : hparams.n_layer_all + 1;
}

So in every case where the fit does not run, or runs and fails, placement is byte-for-byte what 99 produced — the only behaviour that changes is the one that was broken. For models with more than 99 layers it is strictly better, since 99 silently truncated.

Setting gpu_layers explicitly still pins the layer count and still disables the fit, which is the documented escape hatch and an acceptance criterion on the ticket. Device selection is unaffected: the addon resolves CPU-vs-GPU from the device key via --device, never from gpu_layers.

The schema description and the generated SDK contract are updated to match. The contract was edited by hand rather than regenerated: a full contract:export in this checkout also rewrote four unrelated JSON-schema fragments (type: [...] → anyOf, and a tuple's items/minItems/maxItems), drift from a locally-resolved zod version rather than from this change. Worth a separate look if CI's contract:check disagrees.

🧪 Tested

131/132 @qvac/inference brittle tests pass, including the whole config-resolution-e2e suite (16/16, 45 asserts) and llm-plugin-transform (9/9). No test asserted on the removed default; the ones that exercise gpu_layers pass it explicitly.

The exception is native-probe-process.test.js, which fails identically before this change — it spawns child processes through a pnpm-installed bare shim that cannot resolve its darwin-arm64 runtime in this checkout. native-probe-run-isolated-fit.test.js cannot load at all here for the same reason (MODULE_NOT_FOUND: bare-events).

Typecheck, prettier and lunte are clean — verified against a pristine-file baseline, since this checkout has three pre-existing unrelated type errors in safe-fetch.ts and examples/.

⚠️ Breaking

None in behaviour, by the argument above: -1 and 99 produce identical placement whenever the fit is not in play, and an explicit gpu_layers is still honoured.

One nuance worth flagging for reviewers: gpu_layers no longer appears in the config object the SDK hands the addon, so anything reading it back off a resolved config will now see undefined rather than 99. Nothing in this repo does.

Not in scope: EMBED_CONFIG_DEFAULTS.gpuLayers keeps its 99. It has the same shape of problem but was outside this ticket's scope, and embedding models are small enough that the fit rarely has anything to reduce. Worth a follow-up.

🤖 Generated with Claude Code

[github-actions]

License compliance — clean

No new dependency license findings in this PR.

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

• ./docs/website
• ./packages/fabric/test/integration
• ./packages/llm-llamacpp/benchmarks/server
• ./packages/llm-llamacpp/benchmarks/performance
• ./packages/inference-addon-cpp/mobile
• ./packages/asr-ggml/benchmarks/server
• ./packages/embed-llamacpp/benchmarks/server
• ./packages/embed-llamacpp/benchmarks/performance
• ./packages/sdk/e2e
• ./packages/vla-ggml/sim/server
• ./.github/actions/release-merge-guard