Conversation
Chrome cannot be attached to after it starts: the DevTools port only exists if --remote-debugging-port was passed at launch, a second process cannot share a running instance's --user-data-dir (Chrome aborts on the profile lock rather than risk corruption), and Chrome 136+ refuses remote debugging outright when the profile is the default user-data-dir. So this drives its own Chrome against a dedicated profile at ~/.ainxt/chrome-profile, seeded once from the user's real profile so their logins carry over. The everyday browser keeps running, untouched. Uses the workspace's existing tokio-tungstenite; no new third-party crate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
chrome_navigate, chrome_read_page, chrome_click, chrome_type and chrome_screenshot, attached to the built-in browser-use agent — which previously carried a prompt telling it to "use the available browsing tools" and no tools at all. chrome_read_page returns the accessibility tree rather than HTML: it is computed after CSS and JS, so it reflects what the page actually renders, and each node carries a [ref=N] handle that click and type address. Scope classification is deliberate. Reading a page is a read. Navigating is not: the browser carries the user's logged-in cookies, so a plain GET can act on their behalf. chrome_screenshot is a write because save_path can create a file anywhere the user can write — capabilities are static, so a tool is classified by the most privileged thing it can do. chrome_type's description forbids entering credentials, and a test asserts that line stays there so a later edit cannot quietly drop it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Covers why a dedicated profile is unavoidable, the read/write scope reasoning, and the safety notes. Records two findings that are otherwise expensive to rediscover: Google sessions cannot be transplanted by copying files, because Chrome binds them with Device Bound Session Credentials held in the Secure Enclave — the cookies copy and decrypt correctly and Google still rejects them, so the fix is to sign in once inside the ainxt window; and screenshots need no macOS screen-recording permission, since the pixels come from Chrome over CDP rather than the OS capture APIs. Numbered 28 to stay clear of upstream's 25-27 chapter names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A security review of the branch found the tools were not gated at all. ToolScope and is_read_only do not drive the approval prompt — AccessKind does — and the five Chrome ToolInput variants had no arms in `From<&ToolInput> for AccessKind`, so they fell through the catch-all to `Read(None)`, which the manager auto-allows unconditionally. Every Chrome tool ran without a prompt in every permission mode, against a browser holding the user's live sessions, while the tests asserting ToolScope::Write passed. Each tool now maps onto AccessKind explicitly. A file:// navigation is classified as a read of that path, so deny_read_globs and read rules apply rather than being bypassed by a URL. Also from the review: - chrome_screenshot was in the read bucket of kind_allowed and returned is_read_only() == true in the taxonomy, contradicting its own capabilities(). A ReadOnly session — including a forked subagent, where the capability table is the enforcement point — was handed a file-write primitive. Moved to the write bucket. - resolve_save_path applied no confinement: it followed symlinks, silently overwrote existing files, created directory trees anywhere, and honoured any extension, so a screenshot could clobber a source file, a shell profile or a CI workflow. It now normalises `..`, refuses symlinks and existing files, requires an existing parent directory, and accepts only image extensions. - existing_instance() adopted whatever served DevTools on the port and followed the returned WebSocket URL verbatim. A local process could bind the port first and become a man-in-the-middle on every command and every page the agent believed it was reading. It now requires the profile's own DevToolsActivePort to name that port, and refuses any non-loopback endpoint. - Privileged URL schemes (devtools:, chrome:, view-source: and friends) are refused; devtools:// navigation previously succeeded, and those pages can reach the debugging APIs of the browser driving them. - The profile seed copied saved passwords and autofill/card data, which staying logged in does not require. It now copies cookies only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
innajain
changed the title
innajain
changed the title
innajain
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gives the
browser-useagent a browser. It already shipped a prompt telling it to "use the available browsing tools" and no tools at all.chrome_navigatechrome_read_page[ref=N]handleschrome_clickchrome_typechrome_screenshotchrome_read_pagereturns the accessibility tree, not HTML: computed after CSS and JS, so it reflects what actually renders and is far smaller than the DOM. Clicks dispatch real mouse events rather than calling.click(), so hover handlers and event delegation behave normally.Dedicated profile. Chrome can't be attached to after it starts, can't share a running instance's
--user-data-dir, and 136+ refuses remote debugging on the default profile. So this runs its own Chrome against~/.ainxt/chrome-profile, seeded once with cookies only — passwords and autofill are deliberately left behind. Google sessions don't transplant (Device Bound Session Credentials); sign in once in the window instead. Documented, not worked around.Security — read the last commit first. The first cut of this branch was not safe.
ToolScopedoes not gate the approval prompt,AccessKinddoes, and the newToolInputvariants had no arms there — so they fell through toRead(None), which is auto-allowed. Every tool ran with no prompt, in every permission mode, against a browser holding live sessions. Each tool now maps ontoAccessKindexplicitly, andfile://navigation is classified as a read of that path sodeny_read_globsapplies.Also fixed:
save_pathfollowed symlinks, overwrote files and honoured any extension;chrome_screenshotwas read-class, handing aReadOnlysubagent a file write;existing_instance()adopted any DevTools listener and followed its WebSocket URL verbatim;devtools://navigation succeeded.The DevTools port stays unauthenticated on loopback for the session — inherent to CDP, documented, and worth a reviewer's attention if that isn't acceptable here.
Testing. 15 unit tests in
ainxt-chromeplus tool-level tests for scope classification, the credential guardrail, andsave_pathconfinement. Verified end to end against real Chrome on macOS. The 17 failingainxt-workspacetests are pre-existing onmain.I know
CONTRIBUTING.mdsays external PRs aren't triaged. Raising it anyway because thebrowser-useagent currently ships with no browsing tools, and this fills that gap — including a permission-gating bug that would be worth knowing about even if the feature itself isn't wanted. Happy to split it up, rework the approach, or hand it over however is easiest. If there's a better channel for this, point me at it and I'll take it there.