[Bug] An output schema pydantic cannot render raises instead of taking the prompt fallback

[weiqingy]

Search before asking

• I searched in the issues and found nothing similar.

Description

OllamaChatModelConnection.chat translates a BaseModel output schema by calling model_json_schema(). For some field types pydantic cannot produce a JSON Schema and raises PydanticInvalidForJsonSchema, so the chat call fails instead of falling back to the prompt-engineering path.

That is inconsistent with how the same connection treats the other schema form it cannot translate natively. A RowTypeInfo is skipped silently and the caller keeps the prompt fallback, while a BaseModel pydantic cannot render propagates an exception out of chat. Both cases are "this connection cannot express the schema natively", so it is not obvious they should behave differently.

There are two defensible directions, and I do not think the choice is clear cut:

1. Treat an unrenderable BaseModel like any other untranslatable schema and fall back to the prompt path. This makes the two cases consistent and keeps a schema from breaking a call that would otherwise succeed.
2. Keep raising, on the grounds that an unrenderable schema is a caller mistake and a silent fallback would hide it, and instead make the RowTypeInfo case louder.

The choice also affects what "capable" means. The connection reports native support for every model, so the caller has no way to ask in advance whether a particular schema will translate.

Exposure today is limited. No framework path passes an output_schema down to a connection, so this is reachable only from a direct call to the 4-arg chat.

The OpenAI, Azure OpenAI and Anthropic Python connections hand the class to their provider SDK rather than calling model_json_schema() themselves, so they raise from inside the SDK on the same input. Whatever is decided here likely applies to them as well, which is why this is filed against the behavior rather than against one connection.

How to reproduce

from typing import Callable

from pydantic import BaseModel

from flink_agents.api.agents.types import OutputSchema


class Bad(BaseModel):
    cb: Callable[[int], int]


# raises pydantic.errors.PydanticInvalidForJsonSchema
connection.chat(
    messages, model="qwen3", output_schema=OutputSchema(output_schema=Bad)
)

Bad.model_json_schema() on its own raises the same error, which is the call the connection makes.

Version and environment

Flink Agents main (0.3-SNAPSHOT), pydantic 2.11.4, Python 3.10 to 3.12. Not platform specific.

Are you willing to submit a PR?

• I'm willing to submit a PR!

[wenjin272]

I lean toward option 2. A BaseModel that Pydantic cannot render is invalid for structured output, and silently falling back would hide a caller error while weakening the schema guarantee. We should raise a clear, consistent error across connections, while treating RowTypeInfo as a separate, explicitly documented fallback case.

[weiqingy]

Thanks for the feedback @wenjin272.

Option 2 also turns out to be the only workable one. The prompt fallback cannot render this schema either: ReActAgent builds its instruction prompt by rendering the schema (python/flink_agents/api/agents/react_agent.py:130), so an unrenderable model already raises there, before any connection is involved. There is no schema left to put in the prompt.

One correction to my own issue while I am here: the Ollama model_json_schema() call I cited is in #981, not on main.

I will open a PR and we can go through the implementation in review.