fix: enable Qwen3-Max prompt caching for hosted evals

[stu-cao]

Why this change is needed

Hosted Evals launches prime eval run, whose default inference path builds a non-streaming Chat Completions request in OpenAIChatCompletionsClient. Alibaba's qwen/qwen3-max endpoint does not create an explicit prompt cache from provider pinning alone: the request must contain block-level cache_control: {"type": "ephemeral"} breakpoints. See the OpenRouter Alibaba Qwen caching contract.

The inference gateway already preserves caller-authored markers. It should not infer stable prompt boundaries for heterogeneous clients. This client is the narrowest layer where the resolved model, destination API, final structured messages, and ordered tool definitions coexist immediately before the request is sent.

What changes

For the exact qwen/qwen3-max model on the exact openrouter.ai or api.pinference.ai hosts, the default Chat Completions client now adds explicit cache breakpoints to:

• the final cacheable text block in the leading system/developer prefix;
• the final function tool, preserving tool order.

The transformation is copy-on-write. It does not mutate the rollout prompt or tool definitions, and it never marks the changing user/assistant rollout tail. This limits Alibaba's 1.25x cache-write premium to content expected to be reused across examples, parallel rollouts, and tool turns.

The generated marker is exactly {"type": "ephemeral"}. No ttl is sent because Alibaba supports a fixed five-minute cache, not Anthropic's optional one-hour retention.

Caller intent always wins: if cache_control or prompt_cache_breakpoint is already present at the request, message, direct content-block, or tool level, this client leaves the entire request unchanged. Tool parameter schemas containing a property named cache_control are not mistaken for cache directives.

Scope and related changes

• PrimeIntellect-ai/inference-iac#157 controls provider locality and pins Qwen3-Max to Alibaba.
• PrimeIntellect-ai/platform#4450 locks the gateway passthrough contract for caller-authored Qwen markers.
• PrimeIntellect-ai/platform#4447 was closed because gateway-side breakpoint inference and _pi_prompt_cache route metadata put prompt policy at the wrong layer.

This PR intentionally covers the default Hosted Evals Chat Completions path. Explicit Responses, Anthropic Messages, renderer, native v1 proxy, and direct-SDK harness paths are separate request producers and are unchanged.

Validation

• focused client tests: 22 passed;
• offline test suite: 926 passed, 75 API/e2e tests deselected;
• Ruff: passed;
• Python 3.13 type check: passed;
• all pre-commit hooks relevant to the changed Python files passed. The all-files run still reports 321 pre-existing MD033 inline-HTML findings in the untouched legacy SWE README.

No paid inference APIs were called.

Rollout

After merge, publish a verifiers release and ensure Hosted Evals resolves that release. The hosted runner currently installs an upgraded version satisfying the environment's verifiers requirement; exact older pins are intentionally preserved and will not receive this behavior. Post-deploy, verify two back-to-back requests with an identical marked static prefix and require the second response to report nonzero cached prompt tokens.

---

Note

^{Cursor Bugbot is generating a summary for commit 93a9b8a. Configure here.}

Note

Add automatic prompt cache markers for qwen/qwen3-max on hosted eval gateways

• When the model is qwen/qwen3-max and the API base URL is openrouter.ai or api.pinference.ai, OpenAIChatCompletionsClient.get_native_response now auto-inserts cache_control={'type': 'ephemeral'} on the last text segment of the final leading system/developer message and on the last function tool.
• Auto-insertion is skipped if the caller has already supplied any cache_control or prompt_cache_breakpoint markers in the prompt, tools, or extra_body.
• New utility helpers in openai_chat_completions_client.py handle detection and mutation of cache markers without mutating caller-owned objects.
• Behavioral Change: requests for qwen/qwen3-max on the listed hosts will now include prompt cache markers by default unless the caller opts out by providing their own markers.

^{Macroscope summarized 93a9b8a.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR enables new runtime behavior by automatically injecting prompt cache markers into API requests for Qwen3-Max on specific hosts. While the scope is narrow (one model, two hosts), this is a feature enablement with cost implications (1.25x cache-write premium) rather than a simple bug fix, warranting careful review of the caching logic and its integration points.

^{You can add or adjust custom eligibility rules. Learn more.}