Constrained decoding terminates generation when the model emits '#' inside a string value

[AdamMiltonBarker]

Summary

With response_format set to a JSON schema, generation stops when the model attempts to emit a # character inside a string value. The value is cut mid-string and finish_reason is reported as stop, so the truncation is not visible from the response metadata.

Other punctuation is unaffected, and the same input completes normally with no schema, which points at the constrained-decoding path rather than the model.

Environment

• OVMS 2026.3 (ovms_windows_2026.3.0_python_off)
• OpenVINO 2026.3.0-22451-8a17657b995
• Windows 11
• Device: GPU (Intel Arc 140V iGPU, Core Ultra 7 258V)
• Model: OpenVINO/Qwen3.6-35B-A3B-int4-ov

Served with:

ovms.exe --rest_port 8000 \
  --model_path ov_models\qwen3.6-35b-a3b-int4-ov \
  --model_name qwen3.6-35b-a3b \
  --task text_generation \
  --target_device GPU \
  --tool_parser hermes3 \
  --cache_dir .ov_cache \
  --enable_prefix_caching true

Reproduction

Reproduction script: https://github.com/CogniTechSystems/moe-on-arc/blob/main/probe_hash.py

Standard library only, no dependencies, and it imports no project code. Prints the exact request body with --show-request. Exits 1 on reproduction.

python probe_hash.py --repeat 5

Schema is a single required string field:

    {
      "type": "object",
      "properties": { "invoice_number": { "type": "string" } },
      "required": ["invoice_number"],
      "additionalProperties": false
    }

Request parameters: prompt Extract the invoice number from: <input>, temperature: 0, max_tokens: 128, chat_template_kwargs: {"enable_thinking": false}, stream: false.

Results

5 calls per input per mode.

Input	With schema	Without schema
INV-#-99	truncated 5/5, emitted # 0/5	truncated 0/5, emitted # 5/5
INV-%-99	truncated 0/5	truncated 0/5
INV-@-99	truncated 0/5	truncated 0/5
INV-&-99	truncated 0/5	truncated 0/5
INV-X-99	truncated 0/5	truncated 0/5

Controls with other punctuation pass 20/20 under a schema. The # case never emitted the character in any of 5 attempts under a schema, and emitted it in all 5 attempts without one.

Expected vs actual

Expected, and what the same request returns with no schema:

{"invoice_number": "INV-#-99"}

Actual, with schema, finish_reason: "stop", 10 completion tokens:

{
  "invoice_number": "INV-

Notes

• Reproduces identically whether streaming or not. When streaming, the terminating chunk carries "finish_reason":"stop" alongside the partial content.
• Not affected by max_tokens. Output stops at the same character position at 128, 256 and 512.
• Not affected by --enable_prefix_caching, request sequence position, or a cold versus warm --cache_dir. Each was tested and eliminated.
• Only tested on --target_device GPU. Not verified on CPU.

[mzegla]

Thanks for noting that. We have implicit logic that injects '#' as a stop string when structured output generation is requested.
Seems like some legacy code. Will take a look.