Skip to content

Improve browser agent prompt efficiency - #822

Open
thomwebb wants to merge 1 commit into
mpfaffenberger:mainfrom
thomwebb:feat/update-browser-system-prompts
Open

Improve browser agent prompt efficiency#822
thomwebb wants to merge 1 commit into
mpfaffenberger:mainfrom
thomwebb:feat/update-browser-system-prompts

Conversation

@thomwebb

@thomwebb thomwebb commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add assertion scoping, evidence-based waits, route reuse, and single-page execution guidance to qa-kitten
  • add known-route execution, batched extraction, evidence-ledger, and bounded-selector guidance to web-retriever
  • preserve explicit verification for state-changing web workflows
  • add focused prompt-contract tests for both agents

Benchmark provenance

These policies come from a staged browser-agent prompt campaign that explored 300 generations across deterministic local fixtures and read-only live tasks.

The source finalists were:

  • QA candidate: 20/20 matching breadth checks, with repeated medium/high live validation
  • web candidate: 13/15 matching breadth checks, including 3/5 cart checks; stateful cart reliability remains a known caveat

This PR intentionally ports the winning execution policies onto current OSS main rather than copying the finalist prompts byte-for-byte. The benchmark base contained organization-specific guidance and predated the newer OSS confirmation-before-persisting policy. This adaptation removes organization/model-specific wording and retains the newer OSS safeguard.

Validation

  • ruff check --fix and ruff format .
  • focused browser-agent prompt tests: 4 passed
  • full test suite: 7,556 passed, 28 skipped, 1 xpassed
  • deterministic breadth gates against the exact PR prompt hashes: 7/7 passed, including QA visual validation and both stateful-cart checks
  • Python compilation and internal-reference leak scan

Prompt hashes used for the PR-specific breadth run:

  • qa-kitten: 3ec8955b4275e7a0c74dff1913991d5f17064540e384fafab910e71a3bc82cbb
  • web-retriever: 398f66406e435a7328dd49300b19e36547d602452219f74f3e21c26c4f0ae698

Review notes

The prompt additions explicitly take priority over older generic workflow narration where the policies conflict. This avoids requiring routine plan narration, repeated snapshots, workflow writes, or discovery revisits while retaining extra checks when evidence conflicts or an action changes state.

@mpfaffenberger

Copy link
Copy Markdown
Owner

This looks less efficient. Did you meant to replace?

@thomwebb
thomwebb marked this pull request as draft August 21, 2026 02:20
@mpfaffenberger
mpfaffenberger marked this pull request as ready for review August 21, 2026 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants