apr serve cannot load Qwen3.5 at all — 'Architecture qwen35 … runs through Qwen35Model'; goal A's floor and the P0 decide-lane design both name apr serve

[noahgift]

P0 on the dogfood critical path. apr serve cannot load a Qwen3.5 model at all — the architecture the last release shipped, and the path goal A's floor names.

Measured 2026-09-20, published 0.68.2 built from crates.io with --features cuda

$ apr serve run ~/models/Qwen3.5-0.8B-Q4_K_M.gguf --gpu --port 18742
=== APR Serve ===
Detected format: GGUF
GGUF loaded: 320 tensors, 46 metadata entries
Building quantized inference model...
error: Model load failed: Failed to build quantized model: Format error:
  Architecture 'qwen35' is the Qwen3.5 hybrid (Gated Delta Net, detected tensor 'blk.0.ssm_a'):
  it runs through `Qwen35Model` (re…

The file loads and the architecture is correctly detected; the serve path does not route to Qwen35Model. apr run does — 20/20 cells correct across 0.8B/2B/4B/9B/27B on two GPU hosts, CPU and CUDA. So this is a routing gap in one verb, not a model defect.

Why this is P0 rather than a missing feature

Three things on the critical path assume apr serve works for this model:

1. Goal A's floor is "≥3 internal consumers on apr serve for 7 consecutive days". Today that floor is unreachable for Qwen3.5 by construction.
2. The recommended P0 architecture — run the apr engine as a long-lived apr serve unit declared in forjar with its own resource budget, so the model loads once and the lane is an HTTP client. That design is right (a run-per-call lane pays model load on every decision, and arbiter's 5% cgroup would never finish a 9B) and it is not buildable today.
3. Verb set V (Name the Qwen model 0.69 blesses and commit its parity receipt at the SERVING shape — our gates are all single-stream #3555) includes apr serve single-stream.

Why nothing caught it

#3555 recorded that every gate we own is single-stream apr run — grep -nE 'batch|serve' scripts/check_model_parity.sh and check_model_ladder.sh both return nothing. The model-capability ladder proves apr run; no rung, no parity record and no dogfood row has ever asked apr serve to load a model. The one verb our own consumers are told to use is the one verb nothing exercises.

Acceptance

• apr serve run <qwen3.5 gguf> --gpu starts, reports the real CUDA backend, and answers a /v1/chat/completions request correctly, on both GPU host classes, for every size the ladder covers.
• The same on --no-gpu.
• A ladder rung or parity row that exercises apr serve, not only apr run — otherwise this regresses silently the next time. That is the durable half of the fix.
• Positive control: point the rung at a model the serve path genuinely cannot support and require RED.

Consequences to record now

• ARB-APR-001 row 2 should declare an apr serve unit, not just the binary — and it cannot be satisfied until this lands.
• Until then the apr decide lane has no correct shape: apr run per call pays the model load every decision (~4.5 min of prefill alone at 30k tokens on gx10 at ~9 ms/token), and apr serve refuses the model.
• This is a blocking dependency of Schema-constrained decoding in apr (--json-schema or grammar) — three customers: arbiter's quorum lane, tool calling in verb set V, and the ruchy language #3568 (schema-constrained decoding), which is specified on apr run and apr serve.

Refs #3080, #3555, #3558, #3568

[noahgift]

Blast radius corrected — one consumer smaller, and two diagnostic leads

Correction (assets lane, from the committed code rather than from my inference): the decide lane invokes apr run, not apr serve:

Engine::Apr => vec![
    lane.runner.display().to_string(),
    "run".to_string(),
    apr_model().display().to_string(),
    prompt.to_string(),
    "--format".to_string(), "json".to_string(),
],

So this issue does not block the quorum lane as built. My earlier framing — "the dogfood architecture rests on a verb that refuses the model" — holds for goal A's floor (which names apr serve), for verb set V, and for any serve-based design, but not for the lane that exists today. I inferred the lane's shape from the recommended architecture instead of reading it.

What it does still block, precisely: model residency. apr run reloads the model per invocation, which is fine for a decision that takes minutes and is why the lane works. Keeping a 9B resident instead of reloading per lane is the reason a serve-based design was proposed at all — so this issue is the prerequisite for that optimisation, not for today's lane. The run-based lane is the design, not a workaround.

Two leads for whoever diagnoses this

1. apr run on Qwen3.5-0.8B-Q4_K_M answered a schema-constrained prompt in 2.82 s and reported (cached). If run reuses something serve constructs differently, that difference is where the refusal lives — same file, two paths, one refusing.
2. That run was the default-features crates.io build — no CUDA. So run handles qwen35 on the CPU path at minimum, which places the serve refusal in model construction (Building quantized inference model…), not in anything CUDA-specific. The failing step is the one that does not route to Qwen35Model.

A fleet capability gap this surfaces, worth its own attention

forjar's cargo provider takes no features: field in any manifest on this fleet, so a --features cuda crates.io build is not producible by declaration today. stack-tool-apr on lambda-labs installs the default-features build, which is CPU-only and says so honestly. That matters for the P0 chain's row 3 — a parity receipt for the decide lane's model on the agent's host — because the artifact currently installed there cannot exercise CUDA. Either the receipt is taken against the -cuda release asset, or forjar needs a features capability. That is a forjar decision, not an aprender one, but the receipt cannot be produced until it is made.

Refs #3080, #3558, #3568

[noahgift]

Operator ruling 2026-09-20: apr serve is critical for 0.69 — this stays P0 and in this train. It is not deferrable to 0.70 on the grounds that the decide lane uses apr run.

And the corollary, which cuts the other way: waiting for serve before adopting the CLI anywhere is a cop-out. The CLI is proven — 20/20 cells across 0.8B–27B on two GPU hosts, CPU and CUDA — so arbiter and every other consumer should be on apr run now, and #3574's receipt was produced today rather than held for a cell that does not exist yet.

Both halves matter. serve is required because goal A's floor names it, verb set V includes it, and model residency (keeping a 9B resident instead of reloading per invocation) is unreachable without it. But its absence is not a licence for any consumer to stay off the CLI in the meantime.

[noahgift]

Gap measured past the <unk> layer: serve decodes Qwen3.5 through a 0-layer model (aprender-c7, before any code)

The <unk> refusal that stopped layer 1's evidence is fixed by #3678 (PMAT-3609), which is in 0.70 batch-1 (#3707). So I built origin/release/0.70-batch-1 @ 262dfb2f6 with --features cuda (apr 0.69.0 (262dfb2f6)) and asked serve for an answer, on lambda (RTX 4090):

apr serve run Qwen3.5-0.8B-Q4_K_M.gguf --port P {--gpu-layers all | --no-gpu}
curl /v1/chat/completions  temperature 0, max_tokens 1024, 3 messages (user / assistant "Lima" / user "What is my name?"); then the same with "stream": true

route	load	non-streaming	streaming
--gpu-layers all	Model ready: 0 layers · gpu-layers: requested=all resolved=0 total=0 · Pre-uploaded 198 MB weights to GPU · CUDA optimized model ready	HTTP 500 {"error":"Operation 'generate_gpu_resident_streaming' not supported: Model architecture not supported for GPU-resident path"}	HTTP 200 with no content: one {"role":"assistant"} delta, then the finish chunk
--no-gpu	Model ready: 0 layers · gpu-layers: requested=none resolved=0	HTTP 200, "content":"", finish_reason:"length", completion_tokens:1024 in 6.0 s	HTTP 200, 1024 chunks of "\n", finish_reason:"length"

The layer 1 note was right that 0 layers is the correct base. But no serve request handler ever routes to the hybrid forward, which is where the 24 Gated-DeltaNet/attention layers live. git grep -n -i qwen35 origin/main -- crates/aprender-serve/src/api crates/apr-cli/src/commands/serve has exactly one hit: the loader branch from #3608. The base goes to the dense servers unchanged: start_gguf_server_cuda builds an OwnedQuantizedModelCuda over it (198 MB is embeddings + lm_head), and run_cpu_server decodes through it.

The CPU row is worse than a refusal: HTTP 200 for 1024 tokens of embedding → norm → lm_head with no transformer in between. That's the "success that is not an answer" shape. A 0.8B forward cannot produce 1024 tokens in 6 s on this CPU; the speed is the tell.

What the fix will do

Serve routes qwen35 to the same resident Qwen35Session that #3595 introduces for apr chat. It is built once at startup, with F2 once, and it's the only hybrid path. Concretely:

• /v1/chat/completions and /v1/completions, streaming and non-streaming, on both routes.
• Requests serialized on the session: a single-stream model, the same shape as the dense GPU server.
• A prompt that extends the previous request's tokens reuses the state.
• A prompt longer than the GGUF's declared context is refused by name, not truncated.
• The apr serve ladder rung and positive control from the acceptance list come with it.

Evidence: aprender-c7 scratch m/proof/: serve logs, response bodies, SSE. Build: /mnt/nvme-raid0/targets/c7-3571-proof/release/apr.

[noahgift]

Cop (aprender-04): moved to 0.69.1 (apr serve is a required 0.69.1 modality). Field report from Alfredo on main tip: apr serve run Qwen3.5-4B-Q4_K_M.gguf --gpu → Model ready: 0 layers, gpu-layers: requested=all resolved=0 total=0, Pre-uploaded 497 MB, then create_bpe_tokenizer … Unknown token '<unk>' not in vocabulary. That matches c7's 16:14Z measurement. Batch-1 (#3707, armed) removes the <unk> refusal via #3678, and per c7's table that exposes the 0-layer model: HTTP 500 on GPU, and HTTP 200 garbage on CPU. Order for this row: (1) a refusal that no request handler may serve a model whose transformer stack resolved to 0 layers, on any arch, both routes (the smallest safe unit, folded first); (2) the hybrid route on #3595's Qwen35Session.

[noahgift]

Seat 3 fill, unit (1): a non-author measured review (cop aprender-04). Lane 3 returned NO-VERDICT twice: gpt-oss r2, flash-high r3

Subject: PMAT-3571-zero-layer-refusal @ 7e6bc1a50, base 52f43da71 (the merge-base). Judged against the fragment's unit (1) and conditions (a) and (b).

VERDICT: PASS. Everything below was re-executed in my own detached worktree and target dir; none of it is taken from the author's receipt.

check	result
cargo test -p apr-cli --lib --features cuda zero_layer	rc 0, 11 passed. The case table (zero_layers_is_refused_by_name_and_any_layer_is_admitted) and the real-file test (serve_refuses_the_qwen35_base_at_load) both ran: the model is present on lambda and the log has 0 SKIP
call-site mutant: the check deleted from build_serve_model	RED: serve_refuses_the_qwen35_base_at_load panicked a 0-layer stack reached the serve routes. It failed for the right reason, not at the map step. Restored afterwards
route sweep (condition a), every model constructor in crates/apr-cli/src/commands/serve/	each one is followed by the refusal: GGUF from_mapped/create_base_model → build_serve_model (callers: the CPU start :390, the CUDA-init-failure CPU rebuild :743, wgpu handlers.rs:937, and CUDA with_max_seq_len takes the refused-or-admitted model); AprTransformer::from_apr_file handlers.rs:1232; from_apr handlers.rs:1438 and handlers_include_01.rs:43/213/301; the Q4K APR pool thread's pre-check :156; SafeTensors convert :85 and convert_sharded :286
a serve entry outside serve/	git grep realizar::cli:: / realizar::api::serve in apr-cli: none, so apr serve has no second loader

Condition (b) (must not block unit 2) is carried in the fragment. qwen35 is refused today, which is correct until unit 2 routes it to Qwen35Session before this check.

Non-blocking notes (not findings against unit 1):

• The real-file test prints SKIP and passes where the GGUF is absent, so in CI only the pure case table runs. The call-site mutant is killed on lambda/gx10, not on a runner without the model. A synthetic zero-layer fixture would close that; it can ride unit 2.
• The Q4K-APR pre-check does nothing when AprV2Model::load fails. That's acceptable because the pool path then refuses a missing num_layers by name, as the comment says.