Skip to content

fix(sandbox): match Modal region pins through the openrouter/ routing prefix - #99

Open
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1789746575-modal-regions-routing-prefix
Open

devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1789746575-modal-regions-routing-prefix

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

TL;DR

resolveModalRegions now strips the openrouter/ routing prefix before matching MODAL_REGION_PINS, so openrouter/sakana/fugu-ultra lands in a US sandbox like sakana/fugu-ultra does.

What changed?

  • New src/harness/model-slug.ts with stripRoutingPrefix, the logic previously inlined in agent-cli/runner.ts#normalizeAgentModel. normalizeAgentModel is now an alias of it, so existing importers are unaffected.
  • resolveModalRegions matches pins against the stripped slug.
  • Regression tests for the prefixed Fugu slug and for the helper (openrouter/auto stays intact).

Why?

Follow-up to #92. Region selection ran against the raw config model while the agent env ran against the normalized one, so a run configured with the routing-prefixed slug got default placement and the provider 403 the pin exists to avoid. Flagged by Devin Review on openrouter-web#44769.

How to test

bun test src/sandbox/modal-regions.test.ts src/harness/model-slug.test.ts

Expected: resolveModalRegions("openrouter/sakana/fugu-ultra", undefined) returns ["us"].

Reviewer focus

  • src/sandbox/modal-regions.ts: the pin match now goes through stripRoutingPrefix. sandbox/ depends on harness/, not on benchmarks/agent-cli/.

Checklist

  • Tests cover changed behavior
  • Public API or configuration changes are backward compatible, or the break is documented
  • Benchmark changes document dataset provenance and licensing
  • No credentials, private results, or restricted dataset contents are included
  • Documentation is updated where needed

Link to Devin session: https://openrouter.devinenterprise.com/sessions/4a11ad51b948470fbe1b893fd1f275f8
Open in Devin Desktop: https://openrouter.devinenterprise.com/desktop/session/4a11ad51b948470fbe1b893fd1f275f8?variant=devin
Requested by: @ayush-or

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Original prompt from Ayush

SYSTEM:
<latest_message>
Ayush Patel (U0B8L6RNMA9) [ts=1789652370.558029]: &gt; ◦ provider region 403 from the modal sandbox, Fugu Ultra/Max terminal-bench blocked. needs sandbox region pinning or egress through OR. no PR. maybe we just support pinning sandbox region? modal accepts a param
@Devin can you make a pr to enable this and default to US in benchmark-harness / openrouter-web
read modal upstream docs if needed, see the price diff if we region pin
</latest_message>

=== BEGIN THREAD HISTORY (in #agents) ===
Ayush Patel (U0B8L6RNMA9) [ts=1789652370.558029]: &gt; ◦ provider region 403 from the modal sandbox, Fugu Ultra/Max terminal-bench blocked. needs sandbox region pinning or egress through OR. no PR. maybe we just support pinning sandbox region? modal accepts a param
@Devin can you make a pr to enable this and default to US in benchmark-harness / openrouter-web
read modal upstream docs if needed, see the price diff if we region pin
=== END THREAD HISTORY ===
Channel ID: C07UF9XLTFF
Thread URL: https://openrouter.slack.com/archives/C07UF9XLTFF/p1789652370558029?thread_ts=1789652370.558029&amp;cid=C07UF9XLTFF

The <latest_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.
A [ts=...] marker on a Slack message is that message's timestamp. To act on a specific message with the slack tool (e.g. adding an emoji reaction via the reaction command), pass that value as timestamp along with the Channel ID — no extra lookup call is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant