End generation inside the classifier, build grammars and tool parsers once, reuse sampling buffers, and bump to 0.16.0 by mcharytoniuk · Pull Request #34 · intentee/llama-cpp-bindings

End generation inside the classifier, release held tokens without emitting the end token, make bare JSON tool call detection opt-in, and probe JSON incrementally without per-token copies

mcharytoniuk

committed

44539a8

Report whether a model's vocabulary adds a BOS token

mcharytoniuk

committed

affc799

Parse a grammar once when building its sampler and diagnose a rejected grammar against the model vocabulary so token references are accepted

mcharytoniuk

committed

45322c4

Refill a reusable candidate array from a context's logits instead of allocating one per sample

mcharytoniuk

committed

b77df53

Build a chat message parser's tools parser once and parse every message against it, replacing ChatTools and LlamaModel::parse_chat_message

mcharytoniuk

committed

78cbe1a

Let samplers and multimodal input chunks move between threads

mcharytoniuk

committed

1735470

Check every multimodal input chunk against the batch size before evaluating any of them

mcharytoniuk

committed

12bbe82

Exercise the classifier with an ordinary token in the tests that expect one, since some models begin with an end-of-generation token

mcharytoniuk

committed

25c20f0

Bump workspace to 0.16.0

mcharytoniuk

committed

6813975

Reject batches that must fit one micro batch but exceed it instead of letting llama.cpp abort

mcharytoniuk

committed

1e84089

Pass CUDAARCHS to CMake as the CUDA architectures ggml builds for

mcharytoniuk

committed

b1a84cc

Load each test phase's model only when one of its tests runs

mcharytoniuk

committed

c670d2e

Reject non-causal media chunks larger than one micro batch before evaluating any chunk

mcharytoniuk

committed

4f5a457

Take the micro batch from a non-zero batch size and reject a zero batch in every evaluation path

mcharytoniuk

committed

1b38a74

Navigation Menu

Repository navigation

End generation inside the classifier, build grammars and tool parsers once, reuse sampling buffers, and bump to 0.16.0 - #34

End generation inside the classifier, build grammars and tool parsers once, reuse sampling buffers, and bump to 0.16.0#34
mcharytoniuk merged 14 commits into
mainfrom
prepare-0-16-0

Commits on Sep 29, 2026

End generation inside the classifier, release held tokens without emitting the end token, make bare JSON tool call detection opt-in, and probe JSON incrementally without per-token copies

Report whether a model's vocabulary adds a BOS token

Parse a grammar once when building its sampler and diagnose a rejected grammar against the model vocabulary so token references are accepted

Refill a reusable candidate array from a context's logits instead of allocating one per sample

Build a chat message parser's tools parser once and parse every message against it, replacing ChatTools and LlamaModel::parse_chat_message

Let samplers and multimodal input chunks move between threads

Check every multimodal input chunk against the batch size before evaluating any of them

Exercise the classifier with an ordinary token in the tests that expect one, since some models begin with an end-of-generation token

Bump workspace to 0.16.0

Reject batches that must fit one micro batch but exceed it instead of letting llama.cpp abort

Pass CUDAARCHS to CMake as the CUDA architectures ggml builds for

Load each test phase's model only when one of its tests runs

Reject non-causal media chunks larger than one micro batch before evaluating any chunk

Take the micro batch from a non-zero batch size and reject a zero batch in every evaluation path