feat: add search_mixedbread web-search tool to bash harness

[samsja]

Summary

Adds a search_mixedbread web-search tool to the bash harness, alongside the existing Serper search tool. It performs Mixedbread Basic Web Search (POST https://api.mixedbread.com/v1/stores/search with store_identifiers: ["mixedbread/web"]) and returns each result as title, URL, relevance score, and the page-content chunk text (a large chunk of the page — often the whole page for short pages — not just a snippet).

Changes

• verifiers/v1/harnesses/bash/program.py:
  • SEARCH_MIXEDBREAD_TOOL schema (name: search_mixedbread, params: query, num_results → top_k)
  • run_mixedbread_search() — POSTs to Mixedbread with Authorization: Bearer, wraps failures as tool errors
  • format_mixedbread_results() — title / URL / score / full content text per result
  • --mixedbread-search / --mixedbread-key argv plumbing + dispatch
• verifiers/v1/harnesses/bash/harness.py:
  • New search_mixedbread: bool = False config flag
  • MIXEDBREAD_PROMPT system-prompt fragment
  • Key resolution: MIXEDBREAD_API_KEY is popped from the program env and handed over argv (same secret hygiene as the Serper key), so the agent's bash subprocesses never inherit it

Enable

--harness.search_mixedbread true   # + MIXEDBREAD_API_KEY in the eval environment

Verified

• Live-tested against the real API with a key: returns title/URL/score/full content (e.g. a Wikipedia page returned ~5.6K chars of page content; a Substack article ~6K chars)
• Follow-up "full content" endpoints (list-chunks, files.retrieve, grep) return 404 for the web store — the search response's text field is the full content surface
• ruff clean; harness config tests pass

---

Note

^{Cursor Bugbot is generating a summary for commit 0320859. Configure here.}

Note

Add search_mixedbread web-search tool to BashHarness

• Adds a search_mixedbread: bool flag to BashHarnessConfig that enables a Mixedbread Basic Web Search tool, requiring MIXEDBREAD_API_KEY in the program environment.
• Implements the tool in program.py: run_mixedbread_search calls the Mixedbread API via httpx.post, formats results with format_mixedbread_results, and returns error strings instead of raising.
• Passes --mixedbread-search and --mixedbread-key to the program and appends MIXEDBREAD_PROMPT to the system prompt when the flag is enabled, mirroring the existing Serper search pattern.
• Behavioral Change: when search_mixedbread is enabled but MIXEDBREAD_API_KEY is missing, the harness raises ValueError at program launch; without the flag the tool is entirely unavailable.

📊 Macroscope summarized 0320859. 2 files reviewed, 1 issue evaluated, 0 issues filtered, 1 comment posted

🗂️ Filtered Issues

[macroscopeapp]

🔴 Critical bash/harness.py:127

The Mixedbread credential is exposed to the untrusted agent, which can read the program's /proc/$PPID/cmdline or process listing from a bash child and recover MIXEDBREAD_API_KEY. Removing it from env does not protect it because --mixedbread-key places the secret directly in argv; pass the key through a channel inaccessible to agent subprocesses instead.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/harnesses/bash/harness.py around line 127:

The Mixedbread credential is exposed to the untrusted agent, which can read the program's `/proc/$PPID/cmdline` or process listing from a `bash` child and recover `MIXEDBREAD_API_KEY`. Removing it from `env` does not protect it because `--mixedbread-key` places the secret directly in `argv`; pass the key through a channel inaccessible to agent subprocesses instead.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR adds a new search tool capability with external API integration, which warrants human review. Additionally, there is an unresolved Critical security finding regarding credential exposure via command line arguments that needs to be addressed.

Not approved because:

• 1 blocking correctness issue found at or above your repo's Minimum Blocking Severity

^{Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.}