Summary
When generation stops inside a <tool_call> block — normally because it hit the
output limit while writing a large argument — the parser takes the remainder of
the text as the block and emits a complete-looking tool call built from partial
input. The client cannot detect this: the arguments JSON is well formed,
because it is rebuilt from the parsed parameters. A shell heredoc cut mid-body
becomes an unterminated cat <<EOF.
Affects qwen3_5vl, qwen3_6_moe and qwen3_5_omni. Reproduced on main at
92f3f13.
Environment
- OS: Ubuntu 26.04.1 LTS (kernel 7.0.0-31-generic)
- NPU driver: NPU FW 1.1.2.64, amdxdna 0.7
flm version: FLM v1.0.6flm validate:
[Linux] Kernel: 7.0.0-31-generic
[Linux] NPU: /dev/accel/accel0 with 8 columns
[Linux] NPU FW Version: 1.1.2.64
[Linux] amdxdna version: 0.7
[Linux] Memlock Limit: infinity
Reproduce
flm serve qwen3.5:9b
curl -s localhost:52625/v1/chat/completions -H 'Content-Type: application/json' -d '{
"model": "qwen3.5:9b",
"messages": [{"role":"user","content":"Use the bash tool to write a very long 300-line poem to /tmp/p.txt using a cat heredoc."}],
"max_tokens": 400,
"tools": [{"type":"function","function":{"name":"bash","description":"Run a shell command",
"parameters":{"type":"object","properties":{"command":{"type":"string"}},"required":["command"]}}}],
"tool_choice": "auto"
}' | jq '.choices[0]'Observed on 92f3f13:
finish_reason : length
tool_calls : True
args length : 1580
args tail : '...n a sacred soundless gown.\n\nSo let us walk gently on this precious"}'
The heredoc has no closing EOF. A client that executes this hangs waiting for
the terminator.
Cause
parse_nstream_content, e.g. modeling_qwen3_5vl.cpp:493-500:
} else {
// Unclosed tag — search for </function> fallback
size_t func_end_pos = response_text.find(func_end_tag, block_content_start);
if (func_end_pos != std::string::npos) {
block_end = func_end_pos + func_end_tag.length();
} else {
block_end = response_text.length(); // <-- fabricates a complete call
}The </function> fallback is reasonable: a block missing only the outer
</tool_call> is still complete. The final else is not — neither closing tag
present means the call was never finished.
Impact
finish_reason is length, which is the only hint, and clients commonly ignore
it when tool_calls is present. The result is a syntactically valid tool call
carrying a truncated command. For file-writing agents that is an unterminated
heredoc; for other tools it is a silently truncated argument.
Fix
PR follows: drop the block when neither closing tag is present, so no tool call
is emitted. The response still carries finish_reason: "length", which is the
honest signal that the client should retry with a larger budget. Complete calls
and the </function> fallback are unaffected.
Summary
When generation stops inside a
<tool_call>block — normally because it hit theoutput limit while writing a large argument — the parser takes the remainder of
the text as the block and emits a complete-looking tool call built from partial
input. The client cannot detect this: the
argumentsJSON is well formed,because it is rebuilt from the parsed parameters. A shell heredoc cut mid-body
becomes an unterminated
cat <<EOF.Affects
qwen3_5vl,qwen3_6_moeandqwen3_5_omni. Reproduced onmainat92f3f13.
Environment
flm version:FLM v1.0.6flm validate:Reproduce
Observed on 92f3f13:
The heredoc has no closing
EOF. A client that executes this hangs waiting forthe terminator.
Cause
parse_nstream_content, e.g.modeling_qwen3_5vl.cpp:493-500:The
</function>fallback is reasonable: a block missing only the outer</tool_call>is still complete. The finalelseis not — neither closing tagpresent means the call was never finished.
Impact
finish_reasonislength, which is the only hint, and clients commonly ignoreit when
tool_callsis present. The result is a syntactically valid tool callcarrying a truncated command. For file-writing agents that is an unterminated
heredoc; for other tools it is a silently truncated argument.
Fix
PR follows: drop the block when neither closing tag is present, so no tool call
is emitted. The response still carries
finish_reason: "length", which is thehonest signal that the client should retry with a larger budget. Complete calls
and the
</function>fallback are unaffected.