Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions src/providers/registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,29 @@ export function providerMatchesRegistryTransport(
return normalizedProviderEndpoint(provider.baseUrl) === normalizedProviderEndpoint(entry.baseUrl);
}

/**
* `providerMatchesRegistryTransport` for a configured name that may be a generated-metadata
* ALIAS rather than a registry id.
*
* A registry row claims extra names through `extraMetadataAliases` (`gemini` for `google`,
* `anthropic-key` for `anthropic`, ...), and `resolveMetadataProvider` resolves those names —
* case-folded, the way saved provider keys arrive — to the row's metadata bundle. A provider
* saved under an alias is owned by the declaring entry, so its transport must be validated
* against that entry; an id-only lookup finds no `gemini` row and would drop a verdict the
* registry still owns.
*/
export function providerMatchesRegistryTransportOrAlias(
name: string,
provider: Pick<OcxProviderConfig, "baseUrl" | "adapter"> & Partial<Pick<OcxProviderConfig, "authMode">>,
): boolean {
const lower = name.toLowerCase();
const entry = getProviderRegistryEntry(name)
?? PROVIDER_REGISTRY.find(row =>
row.id.toLowerCase() === lower
|| (row.extraMetadataAliases ?? []).some(alias => alias.toLowerCase() === lower));
return entry !== undefined && providerMatchesRegistryTransport(entry.id, provider);
Comment on lines +134 to +139

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Metadata aliases still misclassify custom destinations

A custom gemini, anthropic-key, or case-varied provider is accepted through its metadata owner. Routing preserves that custom transport, so generated capabilities can still divert or strip its images.

Learn more

extraMetadataAliases feeds generated model metadata, but it is not a routing alias. The router looks up registry entries by the exact configured provider name in routedProviderConfig. Therefore, a configured provider named gemini keeps its own adapter and base URL, while this helper finds the google row and applies google's permissive transport rule. The same mismatch affects aliases such as anthropic-key and case-varied registry names.

Example: Configure gemini with adapter: "openai-chat" and baseUrl: "https://operator.example/v1". Routing sends requests to that endpoint unchanged. This helper resolves gemini to google, returns true because google does not set preserveCustomDestination, and a generated text-only verdict can send the image to a sidecar or strip it.

Recommended fix: For names that are not exact registry IDs, require the configured adapter, auth mode, and normalized endpoint to match the resolved alias owner directly. Do not reuse the owner's pinning rule, because routing does not canonicalize extraMetadataAliases. Add regression coverage for a custom alias-named destination and a case-varied custom provider.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +130 to +139

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Description omits alias ownership expansion

The description promises adapter-and-destination matching. The implementation additionally treats metadata aliases and case-varied names as registry-owned, so the stated scope needs revision.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

}

/**
* Resolve the registry entry a configured provider actually points at, by TRANSPORT
* rather than by name.
Expand Down
8 changes: 8 additions & 0 deletions src/vision/eligibility.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import { getModelMetadataCaseInsensitive, resolveMetadataProvider } from "../gen
import { nativeInputModalities } from "../codex/catalog/metadata";
import { SUPPORTED_NATIVE_OPENAI_SLUGS } from "../codex/catalog/native-models";
import { enrichProviderFromRegistry } from "../providers/derive";
import { providerMatchesRegistryTransportOrAlias } from "../providers/registry";
import { isCanonicalOpenAiForwardProvider } from "../providers/openai-tiers-destination";

/**
Expand Down Expand Up @@ -280,6 +281,13 @@ function modelAcceptsImageInputWithCache(
}
const fromRow = advertisesImageInput(candidate.inputModalities);
if (fromRow !== undefined) return fromRow;
// A preset name is not transport identity. Some fixed key presets intentionally preserve a
// same-named custom destination, so vendor metadata is authoritative only while the configured
// adapter and endpoint still belong to the registry row that owns that name — where "owns"
// includes canonical metadata aliases like `gemini` or `anthropic-key`, resolved to the entry
// that declares them. Otherwise the capability is unknown and request dispatch must preserve
// the custom destination's image boundary.
if (provider !== undefined && !providerMatchesRegistryTransportOrAlias(candidate.provider, provider)) return undefined;
return metadataImageInput(candidate.provider, candidate.id);
}

Expand Down
2 changes: 1 addition & 1 deletion structure/runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -452,7 +452,7 @@ declare `modelInputModalities: ["text", "image"]` per model for the nine Claude
explicit operator overrides; unknown models receive no new declaration. Client eligibility filters
and Anthropic image wire handling remain unchanged.

`src/vision/plan.ts` prevents raw image bytes from reaching any target whose effective capability is positively known to exclude image input. Evidence is consulted highest-first: `modelCapabilities`, an explicit custom row for the same routed identity, `noVisionModels`, an explicit per-model modality list without `image`, then backend-specific/registry/vendor metadata. A proven text-only target is preprocessed through the configured Vision Sidecar; a positively image-capable target receives the image directly. Genuinely unknown custom models retain the existing compatibility path rather than being guessed text-only.
`src/vision/plan.ts` prevents raw image bytes from reaching any target whose effective capability is positively known to exclude image input. Evidence is consulted highest-first: `modelCapabilities`, an explicit custom row for the same routed identity, `noVisionModels`, an explicit per-model modality list without `image`, then backend-specific/registry/vendor metadata. Registry/vendor metadata applies to a configured provider only while its adapter and destination still match the transport of the registry row owning its name — canonical metadata aliases such as `gemini` resolve to the entry that declares them; a same-named custom destination remains unknown. A proven text-only target is preprocessed through the configured Vision Sidecar; a positively image-capable target receives the image directly. Genuinely unknown custom models retain the existing compatibility path rather than being guessed text-only.

Canonical ChatGPT Codex forwarding uses the generated `openai-codex` capability bundle rather than the public `openai` bundle. This matters when the two backends differ: for example, the vendored metadata records `gpt-5.3-codex-spark` as text-only on `openai-codex` while the public OpenAI row lists image input. The native Chat fast path and web-search image verbalization consume the same effective-capability decision.

Expand Down
4 changes: 4 additions & 0 deletions tests/adapters/openai/openai-chat-dangling-toolcalls.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ const provider: OcxProviderConfig = {
baseUrl: "https://example.test/v1",
apiKey: "sk-test",
authMode: "key",
// The wire role folds to `system` unless a destination is recorded as accepting
// `developer`; this suite is about tool-result repair ordering, so it declares the
// destination rather than asserting the default.
foldDeveloperRoleToSystem: false,
};

interface ChatMsg {
Expand Down
4 changes: 4 additions & 0 deletions tests/responses/chat-inline-document-bytes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ const chatProvider: OcxProviderConfig = {
adapter: "openai-chat",
baseUrl: "https://gateway.example.internal/v1",
apiKey: "k",
// The wire role folds to `system` unless a destination is recorded as accepting
// `developer`; the document test asserts the role a turn keeps, so it declares the
// destination rather than asserting the default.
foldDeveloperRoleToSystem: false,
};
const anthropicProvider = {
adapter: "anthropic",
Expand Down
55 changes: 55 additions & 0 deletions tests/vision/vision-eligibility.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {
visionEligibleModelOptions,
type VisionCandidateModel,
} from "../../src/vision/eligibility";
import { requiresVisionPreprocessing } from "../../src/vision/plan";

const emptyConfig: Pick<OcxConfig, "providers"> = { providers: {} };

Expand Down Expand Up @@ -189,6 +190,60 @@ describe("vision eligibility core", () => {
expect(modelAcceptsImageInput(config, { provider: "runtime", id: "vision" })).toBe(true);
});

test("11d. vendor metadata does not cross a preserved custom destination boundary", () => {
const provider = {
adapter: "openai-responses",
authMode: "key",
baseUrl: "https://operator-gateway.example/v1",
} as const;
const config = configWithProviders({ "zhipu-bigmodel-responses": provider });
const candidate = { provider: "zhipu-bigmodel-responses", id: "glm-5.3" };

// The generated Z.AI bundle calls glm-5.3 text-only, but this preset explicitly permits a
// same-named custom endpoint. Its images must neither leave for a sidecar nor be stripped.
expect(modelAcceptsImageInput(config, candidate)).toBeUndefined();
expect(requiresVisionPreprocessing(config, provider, candidate.id, candidate.provider)).toBe(false);
});

test("11e. a canonical metadata alias keeps the owning registry row's verdict", () => {
// `gemini` is an extraMetadataAlias of the `google` registry row — a provider saved under
// that name resolves to the google bundle but owns no registry id of its own. An id-only
// transport check dropped the verdict entirely, and a generated text-only model silently
// degraded to "unknown" (eligible, no sidecar).
const provider = {
adapter: "google",
authMode: "key",
baseUrl: "https://generativelanguage.googleapis.com",
} as const;
const config = configWithProviders({ gemini: provider });
const candidate = { provider: "gemini", id: "gemini-live-2.5-flash-preview-native-audio" };

expect(modelAcceptsImageInput(config, candidate)).toBe(false);
expect(requiresVisionPreprocessing(config, provider, candidate.id, candidate.provider)).toBe(true);

// Saved provider keys are case-folded by resolveMetadataProvider; the transport binding
// must fold the same way or a title-cased alias would lose the same verdict.
const folded = configWithProviders({ Gemini: provider });
expect(modelAcceptsImageInput(folded, { provider: "Gemini", id: candidate.id })).toBe(false);
});

test("11f. an alias-named custom destination follows the owning entry's pinning", () => {
// `google` is a name-pinned (non-preserved) preset, so a `gemini` row pointed at a custom
// gateway answers with the same verdict the canonical `google` name returns — the alias
// inherits its owner's transport rule rather than failing open or closed on its own.
const provider = {
adapter: "google",
authMode: "key",
baseUrl: "https://operator-gateway.example/google",
} as const;
const config = configWithProviders({ gemini: provider, google: provider });
const id = "gemini-live-2.5-flash-preview-native-audio";

expect(modelAcceptsImageInput(config, { provider: "gemini", id }))
.toBe(modelAcceptsImageInput(config, { provider: "google", id }));
expect(modelAcceptsImageInput(config, { provider: "gemini", id })).toBe(false);
});

test("12. only the selected Anthropic OAuth provider contributes Anthropic options", () => {
const config = configWithProviders({
anthropic: {
Expand Down
Loading