feat(friendli): fetch model list dynamically from /v1/models - #1152
Open
Lee-Si-Yoon wants to merge 6 commits into
Open
feat(friendli): fetch model list dynamically from /v1/models#1152Lee-Si-Yoon wants to merge 6 commits into
Lee-Si-Yoon wants to merge 6 commits into
Conversation
Convert Friendli from a static provider (4 hardcoded models) to a dynamic provider that fetches the live model list from the public https://api.friendli.ai/serverless/v1/models endpoint at runtime. - Add getFriendliModels() fetcher with zod schema validation - Wire friendli into modelCache, webviewMessageHandler, and dynamicProviders - FriendliHandler loads dynamic models in constructor, falls back to static friendliModels for cold-start and API lag - UI model picker uses routerModels.friendli instead of static list - Add fetcher spec (14 tests) and update Friendli.spec.tsx with ModelPicker mock
Lee-Si-Yoon
requested review from
JamesRobert20,
edelauna,
hannesrudolph,
navedmerchant and
taltas
as code owners
August 5, 2026 05:59
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughFriendli now supports dynamic model discovery. The API fetcher maps live model metadata, the cache and handler provide dynamic models with static fallback, and routing and settings use the merged model data. ChangesFriendli dynamic model discovery
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant FriendliModelAPI
participant getFriendliModels
participant ModelCache
participant FriendliHandler
participant RouterModels
participant FriendliSettings
FriendliModelAPI->>getFriendliModels: return live model data
getFriendliModels->>ModelCache: store normalized ModelInfo records
ModelCache->>FriendliHandler: provide dynamic model metadata
FriendliHandler-->>RouterModels: expose model metadata
RouterModels->>FriendliSettings: provide router models
FriendliSettings->>FriendliSettings: merge router and static fallback models
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/api/providers/fetchers/friendli.ts`:
- Around line 159-160: Update the result handling around
friendliModelsResponseSchema.safeParse so failed validation returns an empty
model list instead of consuming response.data.data; only use result.data.data
when result.success is true. Add a test covering a data array containing an
invalid model and verify no invalid metadata reaches the shared model cache.
- Around line 99-132: Update buildSupportsReasoningEffort so it preserves each
model’s API-provided effort values, excluding only "default"; remove the
unconditional FRIENDLI_EXTRA_EFFORTS merge and REASONING_EFFORT_LEVELS
whitelist. Ensure FriendliHandler accepts and sends values such as "ultracode"
by widening its request type if necessary, without removing valid model
capabilities.
- Line 158: Update the Friendli models fetch in the fetcher containing
axios.get<FriendliModelsResponse> to pass the project’s standard bounded request
timeout configuration, and add a regression test asserting that the /models
request uses that timeout.
In `@src/api/providers/friendli.ts`:
- Around line 108-131: Update the model-selection flow around dynamicModels
loading to track whether dynamic model loading has completed. While loading is
pending, preserve a configured requestedId even when it is absent from
providerModels, using default static metadata temporarily; after loading
completes, retain the existing fallback for IDs missing from both model sets.
Add handler coverage for a dynamic-only configured ID before and after loading
completes.
In `@src/shared/api.ts`:
- Line 193: Update the friendli entry in the provider definition to use the
object type instead of an empty object type assertion, removing the
eslint-disable-line suppression. Then run eslint with --prune-suppressions and
--max-warnings=0 against shared/api.ts to confirm no suppression remains.
In `@webview-ui/src/components/settings/providers/Friendli.tsx`:
- Around line 71-82: The Friendli ModelPicker currently falls back to an empty
model set when routerModels.friendli is unavailable. Update the models prop in
the Friendli component to use the existing friendliModels static fallback,
preserving routerModels.friendli when present.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 946fcf21-58e9-4464-b03b-2ef0d024aef5
📒 Files selected for processing (17)
packages/types/src/__tests__/provider-identifiers.test.tspackages/types/src/provider-settings.tspackages/types/src/providers/friendli.tssrc/api/providers/fetchers/__tests__/friendli.spec.tssrc/api/providers/fetchers/friendli.tssrc/api/providers/fetchers/modelCache.tssrc/api/providers/friendli.tssrc/core/webview/webviewMessageHandler.tssrc/shared/api.tswebview-ui/src/components/settings/ApiOptions.tsxwebview-ui/src/components/settings/constants.tswebview-ui/src/components/settings/providers/Friendli.tsxwebview-ui/src/components/settings/providers/__tests__/Friendli.spec.tsxwebview-ui/src/components/settings/utils/providerModelConfig.tswebview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.tswebview-ui/src/components/ui/hooks/useSelectedModel.tswebview-ui/src/utils/__tests__/validate.spec.ts
💤 Files with no reviewable changes (1)
- webview-ui/src/components/settings/constants.ts
Comment on lines
+99
to
+132
| const FRIENDLI_EXTRA_EFFORTS = ["minimal", "xhigh", "max"] as const | ||
|
|
||
| const REASONING_EFFORT_LEVELS = ["disable", "none", "minimal", "low", "medium", "high", "xhigh", "max"] as const | ||
| type ReasoningEffortLevel = (typeof REASONING_EFFORT_LEVELS)[number] | ||
|
|
||
| function buildSupportsReasoningEffort( | ||
| reasoning: boolean | undefined, | ||
| reasoningOptions: FriendliModel["reasoning_options"], | ||
| ): ModelInfo["supportsReasoningEffort"] { | ||
| if (!reasoning && reasoningOptions === undefined) { | ||
| // Non-reasoning model — omit the field. | ||
| return undefined | ||
| } | ||
|
|
||
| const effortOption = reasoningOptions?.find((opt) => opt.type === "effort") | ||
| if (effortOption && Array.isArray(effortOption.values) && effortOption.values.length > 0) { | ||
| // Controllable reasoning model with a discrete effort enum. Extend the | ||
| // API-provided values with the extra efforts the FriendliHandler uses | ||
| // (minimal/xhigh/max), preserving API order and de-duplicating. | ||
| const merged: string[] = [] | ||
| for (const v of effortOption.values) { | ||
| if (!merged.includes(v)) merged.push(v) | ||
| } | ||
| for (const v of FRIENDLI_EXTRA_EFFORTS) { | ||
| if (!merged.includes(v)) merged.push(v) | ||
| } | ||
| // Drop "default" — it's not a real effort level the handler sends; it's | ||
| // a placeholder the API uses to mean "use the model default". Keeping | ||
| // it in the capability array would let shouldUseReasoningEffort match a | ||
| // settings value of "default" that the Friendli API rejects. | ||
| const filtered = merged.filter((v) => v !== "default") | ||
| return filtered.filter((v): v is ReasoningEffortLevel => | ||
| (REASONING_EFFORT_LEVELS as readonly string[]).includes(v), | ||
| ) |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files =="
git ls-files | rg '(^|/)friendli\.ts$|FriendliHandler|modelCache|reasoning|Friendli' || true
echo "== friendli.ts outline =="
ast-grep outline src/api/providers/fetchers/friendli.ts --view expanded || true
echo "== relevant friendli.ts sections =="
cat -n src/api/providers/fetchers/friendli.ts | sed -n '1,240p'
echo "== FriendliHandler references =="
rg -n "shouldUseReasoningEffort|supportsReasoningEffort|reasoning_effort|reasoning effort|xhigh|ultracode|friendli" -S .Repository: Zoo-Code-Org/Zoo-Code
Length of output: 50377
🌐 Web query:
Friendli chat completions reasoning effort values ultracode API model-specific
💡 Result:
In FriendliAI's API, the reasoning_effort parameter is used to control the extent of internal reasoning performed by supported models before generating an answer [1][2][3]. Higher effort values generally result in more thorough responses but require additional computation time [1][4][5]. The available values for the reasoning_effort parameter are [1][4][5]: - minimal - low - medium - high - xhigh - max - ultracode Key details regarding this parameter include: - Model-Specific Support: The parameter affects reasoning-capable models only, and the specific options available for a given model may depend on that model's implementation [1][3]. - Functionality: It is a relative generation effort level intended to improve quality on complex prompts [4][6][5]. - Usage: It is available in various API contexts, including the Chat Completions API [1][3] and the Messages API (e.g., Anthropic-style messages) [6][5]. For models that support internal reasoning, FriendliAI also provides parameters like reasoning_budget to limit tokens used for internal reasoning, and parse_reasoning to control how reasoning content is returned in the API response [1][7][3]. The ultracode value is explicitly listed as a valid option for reasoning_effort across these schemas, indicating it is a supported setting for applicable models [1][4][6].
Citations:
- 1: https://friendli.ai/docs/openapi/container/chat-completions
- 2: https://friendli.ai/docs/openapi/dedicated/inference/chat-completions
- 3: https://friendli.ai/docs/openapi/model-apis/chat-completions
- 4: https://apis.io/apis/friendliai/friendliai-dedicated-messages-api/
- 5: https://friendli.ai/docs/openapi/serverless/messages
- 6: https://friendli.ai/docs/openapi/model-apis/messages
- 7: https://friendli.ai/docs/guides/reasoning
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== parseFriendliModel tail =="
cat -n src/api/providers/fetchers/friendli.ts | sed -n '236,255p'
echo "== Friendli.ts handler snippet =="
cat -n src/api/providers/friendli.ts | sed -n '1,260p'
echo "== reasoning effort contract references =="
rg -n "reasoning_effort|REASONING_EFFORTS|reasoning efforts|ultracode|supportsReasoningEffort" src packages webview-ui -S --max-count 80 || true
echo "== deterministic check: function on api values including ultracode/default =="
python3 - <<'PY'
from pathlib import Path
text = Path("src/api/providers/friendli.ts").read_text()
# Extract ReasoningEffortLevel literals from type expression.
start = text.index("const REASONING_EFFORT_LEVELS = ") + len("const REASONING_EFFORT_LEVELS = ")
end = text.index("] as const", start) + 8
print(text[start:end])
print("contains ultracode:", "ultracode" in text[start:end])
print("contains xhigh:", "xhigh" in text[start:end])
print("contains max:", "max" in text[start:end])
print("contains minimal:", "minimal" in text[start:end])
PYRepository: Zoo-Code-Org/Zoo-Code
Length of output: 50377
Keep Friendli reasoning options model-specific.
Don’t add "minimal", "xhigh" and "max" to every Friendli effort enum, and change the array filter to preserve API-provided values such as "ultracode", except "default". supportsReasoningEffort feeds the settings UI, and FriendliHandler sends the selected value. Add "ultracode" to the handler request type if needed; do not remove valid model capability values.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/api/providers/fetchers/friendli.ts` around lines 99 - 132, Update
buildSupportsReasoningEffort so it preserves each model’s API-provided effort
values, excluding only "default"; remove the unconditional
FRIENDLI_EXTRA_EFFORTS merge and REASONING_EFFORT_LEVELS whitelist. Ensure
FriendliHandler accepts and sends values such as "ultracode" by widening its
request type if necessary, without removing valid model capabilities.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
- fetcher: return empty list when safeParse fails instead of consuming unvalidated response data - fetcher: add 10s timeout to /models axios request - fetcher: preserve API-provided reasoning effort values verbatim, dropping only "default" and unknown values like "ultracode" instead of merging hardcoded extra efforts - handler: track dynamicModelsLoaded and preserve a dynamic-only requestedId during the initial load window so the first request after construction doesn't silently fall back to the default model - shared/api.ts: use object type instead of eslint-disable suppression - Friendli.tsx: fall back to static friendliModels when routerModels is unavailable so the picker always has selectable models
Add friendli to routerModels expectations and mock sequences — it was added to the handler's provider list but the existing tests weren't updated, causing the mock call order to shift and expectations to miss.
Same issue as webviewMessageHandler spec — routerModels expectations and mock sequences needed friendli added to match the handler's provider list.
…oading - fetcher: test ultracode/unknown effort value dropping + de-duplication - handler: mock getModels, test dynamicModelsLoaded branches — pending load preserves dynamic-only id, completed load falls back to default, dynamic info used when available, rejection sets loaded flag
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/api/providers/__tests__/friendli.spec.ts (1)
587-589: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument or remove the double assertion.
These assertions bypass
FriendliHandlertyping withas unknown as Record<string, unknown>. Use an observable completion assertion where possible. If private-state access is required, isolate the cast in one test helper and add a comment that explains why it is necessary.As per coding guidelines, “Use double assertions only as a last resort and explain them with a comment.”
Also applies to: 613-615, 636-638
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/providers/__tests__/friendli.spec.ts` around lines 587 - 589, Replace the repeated double assertions on FriendliHandler’s private dynamicModelsLoaded state with an observable completion assertion where available. If private-state verification is necessary, centralize the cast in a single test helper and document why it is required, then reuse that helper at the referenced assertions.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/api/providers/__tests__/friendli.spec.ts`:
- Around line 587-589: Replace the repeated double assertions on
FriendliHandler’s private dynamicModelsLoaded state with an observable
completion assertion where available. If private-state verification is
necessary, centralize the cast in a single test helper and document why it is
required, then reuse that helper at the referenced assertions.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 55f917a4-31ae-4448-8c9a-2fa456f1382e
📒 Files selected for processing (2)
src/api/providers/__tests__/friendli.spec.tssrc/api/providers/fetchers/__tests__/friendli.spec.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- src/api/providers/fetchers/tests/friendli.spec.ts
github-actions
Bot
added
the
awaiting-review
…enum Friendli API returns reasoning: true for all models, but only GLM-5.2 has a discrete effort enum (["high","max"]). Other models (DeepSeek-V3.2, MiniMax-M2.5, GLM-5.1, gemma, K-EXAONE) only support on/off thinking toggle via chat_template_kwargs.enable_thinking. Previously these models got supportsReasoningEffort: true (boolean), which made the UI show a full effort dropdown (low/medium/high/...) even though the API ignores reasoning_effort for them. Now they get supportsReasoningBinary: true, which shows a simple on/off checkbox. Also fixes max tokens: all Friendli reasoning models with max_completion_tokens now get supportsMaxTokens: true (the fetcher already did this, but the static fallback also needs it — it already has it, so dynamic + static are now consistent). Handler updated to send enable_thinking + parse_reasoning for binary reasoning models when reasoning is enabled, and nothing when disabled.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Convert Friendli from a static provider (4 hardcoded models) to a dynamic provider that fetches the live model list from the public
https://api.friendli.ai/serverless/v1/modelsendpoint at runtime.The endpoint is public (no auth), same as Vercel AI Gateway and OpenRouter in this repo. The endpoint is now reliable and kept up-to-date, so it can be trusted as the source of truth for the available model list.
fetchers/friendli.ts): zod-validated axios call, maps API fields toModelInfo(pricing, cache, reasoning effort, image modality, deprecation)FriendliHandlerfire-and-forgets a model fetch in constructor;getModel()prefers dynamic data, falls back to staticfriendliModels"friendli"moved todynamicProviders; UI usesrouterModels.friendlifor the model pickerReplaces #1028, which was based on an older main and had become stale.
Summary by CodeRabbit
New Features
Bug Fixes