Skip to content

fix(bedrock): send bytes instead of URLs for PDF and document content - #439

Open
taekop wants to merge 1 commit into
posit-dev:mainfrom
taekop:fix/bedrock-url-content
Open

taekop wants to merge 1 commit into
posit-dev:mainfrom
taekop:fix/bedrock-url-content

Conversation

@taekop

@taekop taekop commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Closes #410.

Since #345, content_pdf_url() and content_document_url() pass the URL through to the provider. The two mantle providers inherit that from AnthropicProvider and OpenAIProvider, but bedrock-mantle can't fetch URLs, so those requests fail with a 400.

BedrockMessagesProvider and BedrockResponsesProvider now swap URL content for a copy with the bytes filled in (via ensure_bytes()) before handing it to the parent class, which then takes the base64 path. On the messages side only ContentPDF needed this, since AnthropicProvider already downloads ContentDocument. This follows the same approach as tidyverse/ellmer#1101.

Remote images have the same problem on mantle, but that predates #345, so I've left it out here.

The new tests mock the download, so they don't hit the network. I don't have Bedrock access, so this hasn't been run against a live endpoint.

@cpsievert

Copy link
Copy Markdown
Collaborator

Nice fix, and thanks for citing tidyverse/ellmer#1101 — that made it much faster to compare approaches here.

One thing I want to flag before merging: the two providers fix this in different ways, and I think that's worth closing before it ships.

BedrockMessagesProvider overrides _as_content_block (an existing per-content-type hook AnthropicProvider already exposes for exactly this kind of subclass override), so the fix is a clean 3-line delegate-to-super. BedrockResponsesProvider doesn't have an equivalent hook to reach into — OpenAIProvider builds request items via a free function (as_input_param) threaded straight through _turns_as_inputs — so the fix there has to override the whole turn-list method and rebuild every turn's content list via model_copy before delegating up, even for a chat that carries no PDF/document content at all.

ellmer doesn't have this asymmetry because as_json dispatches on (provider, content) for every content type uniformly — the AWS overrides for ContentPDF/ContentDocument on both Messages and Responses are the same one-liner shape.

I think chatlas should have that same symmetry. Concretely: pull as_input_param out of _provider_openai.py into a @staticmethod _as_input_param on OpenAIProvider, the same way _as_content_block already exists on AnthropicProvider. Then BedrockResponsesProvider can override just that:

@staticmethod
def _as_input_param(content: "Content", role: "Role") -> "ResponseInputItemParam":
    content = bedrock_materialize_url(content)
    return OpenAIProvider._as_input_param(content, role)

instead of the _turns_as_inputs override.

I tried this out against your branch to check it's a real equivalence, not just a smaller diff:

  • _provider_openai.py change is +87/-87 — a pure relocation of the existing function body into the class, no logic changed.
  • _provider_bedrock.py change becomes +5/-9 (currently +30/-0), and now matches BedrockMessagesProvider._as_content_block's shape exactly.
  • Every existing test in test_provider_bedrock_mantle.py passes unmodified, including the two that call chat.provider._turns_as_inputs([turn]) directly — that method is now inherited rather than overridden, and dispatches to the new hook polymorphically.
  • It also means a mantle Responses chat with no PDF/document content no longer rebuilds its entire turn/content history on every request, which the current _turns_as_inputs override does unconditionally.

Happy to open this as a follow-up/alternative branch if useful rather than asking you to rework it — mostly wanted to raise the asymmetry before it lands, since the next OpenAIProvider subclass that needs a per-content override (Azure, Databricks, etc.) will hit the same wall otherwise.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ChatBedrock() mantle APIs send URL content sources, which Bedrock rejects

2 participants