Skip to content

Fix #985: Handle PydanticInvalidForJsonSchema for unserializable BaseModel - #1032

Open
hulincup wants to merge 3 commits into
apache:mainfrom
hulincup:fix-issue-985-pydantic-json-schema
Open

Fix #985: Handle PydanticInvalidForJsonSchema for unserializable BaseModel#1032
hulincup wants to merge 3 commits into
apache:mainfrom
hulincup:fix-issue-985-pydantic-json-schema

Conversation

@hulincup

Copy link
Copy Markdown
Contributor

Description

Fixes #985

When a BaseModel contains fields that cannot be serialized to JSON schema (e.g., Callable fields), the to_strict_json_schema() function raises PydanticInvalidForJsonSchema. Currently, this exception propagates up and breaks the chat call.

The issue description notes that RowTypeInfo schemas gracefully fall back to prompt engineering when the connection can't translate them natively, but BaseModel schemas that can't be serialized raise an exception instead. This inconsistency should be fixed.

Changes

This PR catches PydanticInvalidForJsonSchema in the three connection types that use to_strict_json_schema() or transform_schema():

  1. OpenAI (openai_chat_model.py):

    • Catches exception in _native_response_format()
    • Falls back to prompt engineering (returns None)
    • Logs a warning message
    • Added logger import
  2. Azure OpenAI (azure_openai_chat_model.py):

    • Catches exception in _native_response_format()
    • Falls back to prompt engineering (returns None)
    • Logs a warning message
    • Updated docstring to document the fallback behavior
  3. Anthropic (anthropic_chat_model.py):

    • Catches exception in _native_output_config()
    • Falls back to prompt engineering (returns None)
    • Logs a warning message
    • Updated docstring to document the fallback behavior

Testing

Added comprehensive test file test_unserializable_output_schema.py with three test cases:

  • test_openai_handles_unserializable_schema(): Verifies OpenAI falls back to prompt when schema cannot be serialized
  • test_azure_handles_unserializable_schema(): Verifies Azure falls back to prompt when schema cannot be serialized
  • test_anthropic_handles_unserializable_schema(): Verifies Anthropic falls back to prompt when schema cannot be serialized

Each test uses a BadModel with a Callable field that cannot be converted to JSON schema, and verifies that:

  1. No exception is raised
  2. The native structured output format is not included in the request (fallback to prompt engineering)

Verification

The fix ensures that:

  • Connections handle unserializable BaseModel schemas gracefully
  • Behavior is consistent with RowTypeInfo fallback
  • Users receive a warning log when fallback occurs
  • Existing functionality for serializable schemas is unchanged

Related Issues

… schemas

When a BaseModel contains fields that cannot be converted to JSON schema
(e.g., Callable fields), connections now gracefully fall back to prompt
engineering instead of raising PydanticInvalidForJsonSchema.

Changes:
- OpenAI: Add exception handling in _native_response_format()
- Azure: Add exception handling in _native_response_format()
- Anthropic: Add exception handling in _native_output_config()
- Add warning logs when falling back to prompt engineering
- Update Anthropic docstring to document the fallback behavior
- Add comprehensive test coverage for all three connections

Fixes apache#985
@github-actions github-actions Bot added doc-label-missing The Bot applies this label either because none or multiple labels were provided. fixVersion/0.4.0 priority/major Default priority of the PR or issue. labels Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-label-missing The Bot applies this label either because none or multiple labels were provided. fixVersion/0.4.0 priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] An output schema pydantic cannot render raises instead of taking the prompt fallback

1 participant