Skip to content

Fixing content parsing for onyx #4448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dtrawins merged 2 commits into from
Aug 14, 2026
Merged

Fixing content parsing for onyx #4448

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
41d0fe7
fixing to=user<|message|> in content with new version of model
przepeck Aug 13, 2026
9cf1d6e
fix zlib source (#4449)
dtrawins Aug 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion WORKSPACE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ http_archive(
build_file = "@mediapipe//third_party:zlib.BUILD",
sha256 = "bb329a0a2cd0274d05519d61c667c062e06990d72e125ee2dfa8de64f0119d16",
strip_prefix = "zlib-1.3.2",
url = "http://zlib.net/fossils/zlib-1.3.2.tar.gz",
url = "https://github.com/madler/zlib/releases/download/v1.3.2/zlib-1.3.2.tar.gz",
)

# RapidJSON
Expand Down
3 changes: 2 additions & 1 deletion src/llm/io_processing/onyx/onyx_tool_parser.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ const std::string OnyxToolParser::PARAMETER_NAME_TAG = "<atem:parameter name=\""
const std::string OnyxToolParser::PARAMETER_END_TAG = "</atem:parameter>";
const std::string OnyxToolParser::NAME_ATTR_END_TAG = "\">";
const std::string OnyxToolParser::ASSISTANT_PREFIX = "<|start|>assistant ";
const std::string OnyxToolParser::CONTENT_START_INDICATOR = "to=user<|message|>";
const std::string OnyxToolParser::CONTENT_START_INDICATOR = "to=user";
const std::string OnyxToolParser::MESSAGE_TAG = "<|message|>";
const std::string OnyxToolParser::END_OF_TURN_TAG = "<|eot|>";
Comment thread
przepeck marked this conversation as resolved.

// Static empty map the default-constructed impl binds its const-ref member to (used by the
Expand Down
5 changes: 3 additions & 2 deletions src/llm/io_processing/onyx/onyx_tool_parser.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,8 @@ class OnyxToolParser : public BaseOutputParser {
static const std::string PARAMETER_END_TAG; // "</atem:parameter>"
static const std::string NAME_ATTR_END_TAG; // "\">" -- closes an invoke/parameter name
static const std::string ASSISTANT_PREFIX; // "<|start|>assistant "
static const std::string CONTENT_START_INDICATOR; // "to=...<|message|>"
static const std::string CONTENT_START_INDICATOR; // "to=user"
static const std::string MESSAGE_TAG; // "<|message|>"
static const std::string END_OF_TURN_TAG; // "<|eot|>"
Comment thread
przepeck marked this conversation as resolved.

private:
Expand Down Expand Up @@ -178,7 +179,7 @@ class OnyxToolParser : public BaseOutputParser {
return true;
}
const std::vector<std::string>& getSpecialTagsToErase() const override {
static const std::vector<std::string> specialTagsToErase{ASSISTANT_PREFIX, CONTENT_START_INDICATOR, END_OF_TURN_TAG};
static const std::vector<std::string> specialTagsToErase{ASSISTANT_PREFIX, CONTENT_START_INDICATOR, MESSAGE_TAG, END_OF_TURN_TAG};
return specialTagsToErase;
}
};
Expand Down
2 changes: 2 additions & 0 deletions src/test/llm/output_parsers/onyx_output_parser_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -673,6 +673,8 @@ TEST_F(OnyxOutputParserTest, StreamingReasoningThenContent) {
TEST_F(OnyxOutputParserTest, StreamingContentOnly) {
int i = -1;
std::vector<std::tuple<std::string, ov::genai::GenerationFinishReason, std::optional<std::string>>> chunkToDeltaVec{
{"<|start|>", ov::genai::GenerationFinishReason::NONE, std::nullopt},
{"assistant ", ov::genai::GenerationFinishReason::NONE, std::nullopt},
{"to=user", ov::genai::GenerationFinishReason::NONE, std::nullopt},
{"<|message|>", ov::genai::GenerationFinishReason::NONE, std::nullopt},
Comment thread
przepeck marked this conversation as resolved.
{"Your ", ov::genai::GenerationFinishReason::NONE, R"({"delta":{"content":"Your "}})"},
Expand Down