Skip to content

fix(runtime): atob WHATWG base64 - #1875

Open
izayl wants to merge 3 commits into
rivet-dev:mainfrom
izayl:fix/base64-atob-parity
Open

fix(runtime): atob WHATWG base64#1875
izayl wants to merge 3 commits into
rivet-dev:mainfrom
izayl:fix/base64-atob-parity

Conversation

@izayl

@izayl izayl commented Jul 27, 2026

Copy link
Copy Markdown

What

  • Align fallback atob / btoa behavior with Node/Web globals.
  • Normalize WHATWG forgiving-base64 input before passing it to base64-js.
  • Reject base64url alphabet characters (-, _) for atob.
  • Preserve InvalidCharacterError shape, including legacy code = 5.
  • Add native-sidecar conformance coverage for padded, unpadded, whitespace, invalid padding, base64url, and btoa invalid-character cases.

Impact

I hit this building an app on @rivet-dev/agentos 0.2.14 (macOS arm64) that runs the
pi agent (@agentos-software/pi) inside a VM with the openai-codex provider.
Every prompt failed, with the agent returning a single opaque line:

Failed to extract accountId from token

The credentials were fine. The same ~/.pi/agent/auth.json works when I run pi directly
on the host; inside the VM the token is intact (not expired, chatgpt_account_id claim
present) — I verified that by reading the file back out of the VM. The agent is simply
unable to decode it.

The failing path: pi decodes its OAuth access token on every request to derive the
chatgpt-account-id header, using atob(token.split(".")[1]). Per RFC 7515 §2, JWT
segments are base64url with padding stripped, so that payload's length is rarely a
multiple of 4 (mine: 1351 bytes, % 4 === 3).

This makes the agent completely unusable inside a VM, while working fine on the host —
which is a confusing failure mode to debug, since nothing about the credentials, the
config, or the agent differs between the two.

Root cause

atob is a WHATWG API, not an ECMAScript one, so the bare V8 runtime does not provide it
and the bridge polyfills it in
packages/build-tools/bridge-src/builtins/process.ts#L1162-L1173
by delegating straight to the vendored base64-js (bridge-src/vendor/buffer.ts, imported
on line 12). base64-js implements RFC 4648 §4, so the polyfill deviates from
forgiving-base64 decode in both
directions:

  • Too strictgetLens() (vendor/buffer.ts#L22-L25) throws
    Invalid string. Length must be a multiple of 4 on unpadded input. The spec only fails
    at length % 4 === 1. This is the bug I hit.
  • Too laxvendor/buffer.ts#L20-L21 maps -/_ to 62/63, so base64url input is
    accepted where the spec requires rejection.

Reproduction

Inside any agentOS VM:

atob("eyJhIjoxfQ")    // → Error: Invalid string. Length must be a multiple of 4
atob("eyJhIjoxfQ==")  // → {"a":1}
Both return {"a":1} on Node, Bun, and in browsers. The runtime reports
process.versions.node = 22.0.0, so agents reasonably assume Node-compatible globals.

Validation

  • pnpm --dir packages/build-tools run build:v8-bridge
  • AGENTOS_BUILTIN_CONFORMANCE_CASE=global_base64 cargo +1.91.1 test -p agentos-native-sidecar --test builtin_conformance __builtin_conformance_case_runner -- --nocapture
  • pnpm --dir packages/build-tools test
  • git diff --check

izayl added 2 commits July 27, 2026 17:32
Fallback atob/btoa should match Node and Web error behavior so guest code sees InvalidCharacterError for malformed input instead of runtime-specific decoder behavior. The conformance case locks the guest V8 globals against host Node.
Document why the fallback atob path cannot pass input directly to base64-js and cover the WHATWG unpadded-input behavior alongside rejected base64url characters.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba0071436a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/build-tools/bridge-src/builtins/process.ts Outdated
Only synthesize padding for unpadded forgiving-base64 input. Inputs that already contain padding must be complete and correctly positioned, matching WHATWG and Node atob behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant