feat(settings): the model field is an open picker, sourced from the harness (#493)

[defangdevs]

Summary

Closes the last unchecked box on #493's layout checklist: "if possible, make model input a picker too (but model IDs change often so not sure if we can dynamically inspect?)". The other five went in with #571; I verified each against master and checked them off in the issue before starting this.

Both halves of that question are right, and they pull opposite ways. A closed <select> would break a model ID this box has never heard of (untypeable) and a profile that already names one (a Save with nothing touched would erase it). So the control is free text over a popup — a picker that still accepts anything. That is the rule render_effort_options() already keeps for a value its own list does not know, one control along.

The popup is the page's own, not the browser's. This shipped as an <input list> over a <datalist> first, and that was wrong for a reason no test could see: a <datalist> popup is drawn by the browser, so no rule in settings.css reaches inside it, and on a real browser the suggestions arrived as near-black text on a dark ground. Declaring color-scheme: dark did not change what it drew (tried, then reverted — the commits are on the branch as the record). The box already had the answer: the Starting folder field on "Add session" (issue #131) is a combobox that owns its popup outright — a <ul class="ac"> the page styles like everything else — and has looked right the whole time. The model field is now the same control with a different source.

The two differ in three places and nowhere else: where entries come from (a fetch per directory level, versus a JSON blob the daemon already put on the page), what a keystroke filters (the fragment after the last /, versus the whole value), and what picking one means (append <name>/ and reveal the next level, versus fill the field and close). acKind() names which is which; the popup, the CSS, and the keyboard and mouse handling are shared untouched.

That also removed a moving part. There is no list attribute to keep in step with the assistant picker beside it, so the stale-datalist review finding is gone by construction along with syncModelLists() and the two call sites that existed to defend it — the row's assistant is read at the moment the popup opens.

What can be inspected is the alias. Aliases are the stable half of the churn — opus and sonnet outlive the dated IDs behind them, which is why the harnesses publish them at all. So the list is read off the harness and never written down here; a box learns a new alias the day its harness updates, and there is no table for us to let rot.

Two sources, neither of which can go stale on its own:

1. The harness's own --help. claude names them in the --model option's description (an alias for the latest model (e.g. 'fable', 'opus', or 'sonnet') or a model's full name (e.g. 'claude-fable-5')). codex documents none — its --model says only "Model the agent should use", and codex completion bash file-completes the value — so it contributes nothing rather than us guessing a list on its behalf and shipping it stale.
2. Every MODEL already saved in a profile naming that harness. Free, exactly the set this box actually uses, and the only source a harness with a silent --help has at all.

Only the --model option's own block is harvested. claude quotes an example in --fallback-model and modes in --sandbox; offering a sandbox mode as a model is a suggestion that starts nothing.

The binary comes from CONNECT_BINS via connect_flow() — which is also where the owner's call on #493 put the harness list ("should match the list of harnesses in the connections UI") — so a harness the box has not installed yet is probed no more than its card is, and a probe that fails costs a suggestion rather than the field. One fork per harness per --help change, cached on the binary's (mtime, size); measured at 0.22s cold, ~0.2ms warm.

A saved profile's row names its harness, so the daemon renders that row's list itself; "New profile" preselects nothing (#493 removed the box default), so settings.js follows that row's assistant picker. model_hint_lists() is the single answer to which datalists the page carries, so a row can never reference one that is not there.

User-visible effect

The model field gains a dropdown of its assistant's aliases and of the models this box already uses. It is still free text: anything typeable before is typeable now, and no stored value can be erased by a Save that did not touch it. No new configuration, no new environment variable, nothing to migrate.

Security

No new surface. Suggestion values are html.escape()d and, from the --help source, constrained to [A-Za-z0-9._-] before that. Nothing new is executed: --help on a binary the connections panel already runs auth status on, resolved through the same os.access(X_OK) guard. No profile value is exposed that the panel did not already render — MODEL was already on the page in every row's edit form.

That is a real headless render of the live page — which is itself a small argument for the change, since the browser's own popup could never appear in one. What the daemon puts on the page:

<script type="application/json" id="model-hints">{"claude": ["fable", "opus", "sonnet",
"claude-fable-5"], "codex": ["gpt-5.1-codex-max"]}</script>

The panel with nothing open, showing the row layout the rest of #493's checklist asked for:

Test plan

• python3 tests/test-profile-panel.py — 54 tests, 12 new, green.
• All 34 native checks on aarch64-linux (nix build --keep-going over nix eval .#checks.aarch64-linux --apply builtins.attrNames), exit 0 — profile-panel, golden-snapshot, module-generated-up-to-date, backend-parity, one-spec-both-backends included. Status captured without a pipe.
• nix run .#assemble + nix run .#update-golden committed. settings-daemon.py/.js/.css are web-only payloads, so tests/native/expected/ does not move.
• The popup itself, over CDP on the live page. Opened it, typed to filter (s -> sonnet), switched the assistant and watched the list swap with no attribute in between (claude aliases -> gpt-5.1-codex-max), clicked a suggestion (fills the field, closes the popup), drove it from the keyboard (two ArrowDowns highlight opus, Enter accepts it and does not submit the form), and confirmed a saved row drives its own popup from its own assistant.
• Real browser, real binaries, over CDP. Ran the assembled daemon on a private rig against this box's actual claude/codex, then drove the live page in headless chromium over the DevTools protocol - not a curled copy - so every assertion below went through the page's own code: new pane starts with no list; clicking "Add profile" and picking claude sets pmodel-claude; picking codex sets pmodel-codex; a harness whose datalist is absent removes the attribute; both saved rows carry their own harness's list from the server; and a real Save posts, collapses the editor, resets the form and leaves list=(none) under the restored "Choose an assistant" prompt (finding 1 below), with the new row coming back as sonnet=pmodel-claude and the datalists refreshed inside the morphed #profiles-list.
• Checked the four page-copy test files (webhook.nix, connect.nix, settings-page.nix, sessions-web.nix) and tests/e2e/ for assertions over the MODEL input's attribute run before inserting data-model-input — none match it.

Review

/code-review high over the whole branch. Two findings, both real, both fixed in dfc3b2b with a regression test each:

1. Stale datalist under a picker that no longer names it. The list was synced only on change, which is not the only way a picker's value moves: form.reset() after a save restores the "Choose an assistant" prompt while leaving the attribute set, and Firefox restores a <select>'s value across a plain reload without firing anything. Fixed with a sync at init and after a post — and then made unreachable by the popup rewrite, which reads the assistant when the popup opens and keeps no attribute at all.
2. A transient probe failure pinned for the life of the daemon. harness_model_aliases cached negative results under the binary's own stamp, which never moves for an unchanged binary - so one timeout under load, or a race with a lazy nix profile add still linking the CLI, meant that harness offered nothing until the daemon restarted. A probe that ran and named nothing (codex) is still a real answer and still cached forever; a probe that could not be made is now held only long enough not to re-fork on every render of a per-second live feed.

No security-review trigger: this touches no auth, secret, token, webhook routing, sudo rule or Caddy config. No AWS, IAM, networking or migration impact.

Notes

• One test-isolation bug found and fixed on the way in: connect_flows() expands ~ at call time, and daemon_with() restores the process environment once the import is done — so a profile-panel test that pinned no binary for a harness silently probed the real CLI on the machine running the suite. ProfileFixture.in_fixture() keeps the probe inside the fixture, and says why.
• Server/daemon mode belongs with the pseudo profiles, not as a profile field #504 (server/daemon mode as its own pseudo-profile) stays split out and unchecked on Profiles, finishing touches #493 on purpose.

Fixes #493

🤖 Generated with Claude Code

https://claude.ai/code/session_01NLLSUAnVUAgw7r4JUvbMYV

[coderabbitai]

Warning

Review limit reached

Next included review available in 8 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: c8760e43-d2ff-41ea-9370-bca97d95fa64

📥 Commits

Reviewing files that changed from the base of the PR and between 0cc18b8 and 0ee779c.

📒 Files selected for processing (6)

• modules/agent-box.nix
• modules/src/settings-daemon.py
• modules/src/settings.css
• modules/src/settings.js
• tests/golden/web/payloads/agent-box-settings/bin/agent-box-settings
• tests/test-profile-panel.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}