Run notebook sandboxes with Pixi - #10697
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
All contributors have signed the CLA ✍️ ✅ |
7ff0e9a to
a10835c
Compare
a10835c to
3429bc7
Compare
3429bc7 to
86b14ad
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Pixi provisioning uses an unsupported command and several launch paths lose required backend, dependency, or isolation guarantees.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Introduces session-owned notebook sandboxes with uv and Pixi backends, including CLI selection, package APIs, environment lifecycle, and frontend integration.
Changes:
- Adds
NotebookSandboxand uv/Pixi backend adapters. - Propagates sandbox context through sessions, APIs, OpenAPI, and package UI.
- Adds broad backend, lifecycle, CLI, and frontend tests.
File summaries
| File | Description |
|---|---|
marimo/_environments/sandbox.py |
Defines the sandbox interface and lifecycle. |
marimo/_environments/backends.py |
Implements uv and Pixi adapters. |
marimo/_environments/pixi.py |
Adds Pixi command integration. |
marimo/_environments/uv.py |
Adds exact-command reporting. |
marimo/_environments/environment.py |
Propagates command callbacks. |
marimo/_environments/script_metadata.py |
Adds manifest materialization and copying. |
marimo/_utils/uv_tree.py |
Parses uv/Pixi dependency trees. |
marimo/_config/settings.py |
Stores the selected sandbox backend. |
marimo/_cli/cli.py |
Adds backend-aware sandbox flags. |
marimo/_cli/sandbox.py |
Resolves and launches selected backends. |
marimo/_cli/tips.py |
Handles explicit negation flags. |
marimo/_session/types.py |
Exposes session sandbox state. |
marimo/_session/session.py |
Binds, rebinds, and closes sandboxes. |
marimo/_session/managers/ipc.py |
Launches IPC kernels through sandboxes. |
marimo/_session/app_host/pool.py |
Dispatches app hosts through backends. |
marimo/_runtime/packages/sandbox_package_manager.py |
Adapts sandboxes to package APIs. |
marimo/_runtime/packages/package_managers.py |
Creates sandbox-backed managers. |
marimo/_runtime/packages/pypi_package_manager.py |
Removes legacy uv script mode. |
marimo/_runtime/callbacks/packages.py |
Preserves sandbox manager selection. |
marimo/_server/models/packages.py |
Adds package installation contexts. |
marimo/_server/api/endpoints/packages.py |
Returns sandbox-aware package data. |
packages/openapi/api.yaml |
Documents the context union. |
packages/openapi/src/api.ts |
Regenerates TypeScript API types. |
frontend/src/components/editor/chrome/panels/packages-panel.tsx |
Adds sandbox-specific package UI. |
frontend/src/core/wasm/bridge.ts |
Supplies WASM package context. |
frontend/src/__mocks__/requests.ts |
Updates package response mocks. |
tests/_environments/test_sandbox_interface.py |
Tests sandbox lifecycle and mutations. |
tests/_environments/test_pixi.py |
Tests Pixi commands and environments. |
tests/_environments/test_uv.py |
Tests uv reporting and tree parsing. |
tests/_utils/test_uv_tree.py |
Tests dependency marker parsing. |
tests/_cli/test_sandbox.py |
Tests backend resolution and flags. |
tests/_session/app_host/test_app_host.py |
Updates app-host backend mocks. |
tests/_server/api/endpoints/test_packages.py |
Tests package response contexts. |
tests/_runtime/packages/test_package_managers.py |
Tests sandbox manager creation. |
frontend/src/components/editor/chrome/panels/__tests__/packages-panel.test.tsx |
Tests sandbox package UI behavior. |
Review details
Suppressed comments (1)
marimo/_runtime/packages/sandbox_package_manager.py:79
- Removal has the same restart-state loss:
_sandbox.remove()can synchronize to a different environment after a rebind, but itsEnvironmentChange.requires_restartis discarded and the API returns success while the kernel keeps using the old interpreter. Carry this state through the session/package response or require an explicit restart.
await asyncio.to_thread(self._sandbox.remove, package)
return True
- Files reviewed: 35/35 changed files
- Comments generated: 8
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| editable_paths = [ | ||
| requirement.removeprefix("-e ").strip() | ||
| for requirement in overlay | ||
| if requirement.startswith("-e ") | ||
| ] |
| completed = subprocess.run( | ||
| [require_pixi_bin(), "install", "--help"], | ||
| capture_output=True, | ||
| text=True, | ||
| timeout=10, | ||
| ) | ||
| if completed.returncode != 0 or "--script" not in completed.stdout: | ||
| raise PixiUnsupportedVersionError() |
| if not path.endswith(".py"): | ||
| return | ||
| if not os.path.exists(path) or os.path.getsize(path) == 0: | ||
| return |
| env = dict(os.environ if base_env is None else base_env) | ||
| env.pop("VIRTUAL_ENV", None) | ||
| env["CONDA_PREFIX"] = environment.root | ||
| env["CONDA_DEFAULT_ENV"] = os.path.basename(environment.root) | ||
| bin_dir = os.path.join(environment.root, "bin") | ||
| path = env.get("PATH") | ||
| env["PATH"] = bin_dir if not path else bin_dir + os.pathsep + path |
86b14ad to
648b8ce
Compare
b5db866 to
ebda262
Compare
ebda262 to
701a067
Compare
ddcc8ad to
76e66f8
Compare
Sandbox selection previously had one implementation: uv. The notebook-owned
lifecycle introduced below already names operations in terms of a manifest and
its realized environment; this change supplies those operations with Pixi and
makes the choice explicit at the CLI:
marimo edit --sandbox notebook.py # uv
marimo edit --sandbox=pixi notebook.py # pixi
NotebookSandbox(source, "pixi")
-> PixiBackendAdapter
-> add / remove / sync / packages / launch
Pixi reads the same PEP 723 metadata. Package changes use Pixi script commands,
synchronization recovers the interpreter and environment root from structured
output, and package inspection reads pixi list JSON. Launches layer the running
marimo through pixi exec so the shared manifest does not pin marimo to the
machine that opened it.
Selecting one manager does not import or probe the other. Pixi rejects Python
interpreter overrides, which its script environments cannot honor, and falls
back cleanly for app-host notebooks without script metadata. CLI, adapter, and
fixture tests cover both managers; the guide names the remaining Pixi
limitations.
Preserve the bare `--sandbox` spelling as uv while accepting an explicit Pixi backend. Only fall back to the host interpreter when inline metadata is missing, so failed Pixi solves cannot silently escape the requested sandbox. Expose conventional conda paths without implying support for Pixi activation scripts, and keep conda-only packages out of unsupported PyPI actions.
76e66f8 to
d878c7d
Compare
This PR remains draft with the Pixi layer it builds on. It can be
reviewed now, but depends on a Pixi release containing the merged
upstream support.
The packages panel previously inferred a sandbox from a synthetic
`<root>` tree and fetched the installed-package list separately. That
hid which environment manager owned package changes and presented
configured-manager controls even though additions update the notebook
manifest.
The dependency-tree response now names that context directly:
```py
PackageInstallationContext = (
SandboxPackageContext(backend="uv" | "pixi")
| PackageManagerContext(name=str)
)
DependencyTreeResponse(tree, context)
```
For a sandbox, the panel uses the dependency tree as its package view,
labels the selected backend, explains that additions update inline
metadata, and does not offer an unrelated package-manager setting.
Non-sandbox environments keep their existing list and tree views.
uv returns its full tree with repeated nodes marked as already
displayed. Pixi reads its tree output and keeps the PyPI packages that
the panel can change, leaving conda runtime packages out of the
actionable view. The OpenAPI and WASM clients carry the same tagged
context, and panel tests cover both contexts and an unavailable tree.
This PR remains draft until a Pixi release includes [prefix-dev/pixi#6648](prefix-dev/pixi#6648). The required upstream change has merged. The notebook-owned lifecycle in the previous PR describes operations in terms of a manifest and its realized environment. This PR supplies those operations with Pixi and makes the choice explicit at the CLI: ```console marimo edit --sandbox notebook.py # uv marimo edit --sandbox=pixi notebook.py # pixi ``` ```text NotebookSandbox(source, "pixi") -> PixiBackendAdapter -> add / remove / sync / packages / launch ``` Pixi reads the same PEP 723 metadata. Package changes use Pixi script commands, synchronization recovers the interpreter and environment root from structured output, and package inspection reads `pixi list` JSON. Launches layer the running marimo through `pixi exec`, so the shared manifest does not pin marimo to the machine that opened it. Selecting one manager does not import or probe the other. Pixi rejects Python interpreter overrides, which its script environments cannot honor, and falls back cleanly for app-host notebooks without script metadata. CLI, adapter, and fixture tests cover both managers; the guide documents the remaining Pixi limitations.
Sandbox launches, package changes, and package inspection previously
reconstructed the notebook path and environment through separate
helpers. An install could update the live environment independently from
the PEP 723 manifest, and a rename left later operations to recover that
relationship.
`NotebookSandbox` now owns the source binding and latest environment:
```py
class NotebookSandbox:
launch(...) -> ProcessPlan
add(...) -> EnvironmentChange
remove(...) -> EnvironmentChange
packages() -> PackageState
rebind(source) -> None
```
Each mutation edits the manifest, synchronizes it once, and records the
resulting environment. A bare requirement is pinned to the resolved
version after synchronization; an upgrade reopens an existing exact pin
before solving.
Single-file launches, IPC kernels, and package endpoints keep the same
object for one session. `BackendAdapter` supplies the
environment-manager operations; this PR implements them with uv. Runtime
overlays and fallback launches remain explicit.
Includes squashed changes from:
- [#10697: Run notebook sandboxes with
Pixi](#10697)
- [#10729: Show sandbox context in the packages
panel](#10729)
This PR remains draft until a Pixi release includes prefix-dev/pixi#6648. The required upstream change has merged.
The notebook-owned lifecycle in the previous PR describes operations in terms of a manifest and its realized environment. This PR supplies those operations with Pixi and makes the choice explicit at the CLI:
Pixi reads the same PEP 723 metadata. Package changes use Pixi script commands, synchronization recovers the interpreter and environment root from structured output, and package inspection reads
pixi listJSON. Launches layer the running marimo throughpixi exec, so the shared manifest does not pin marimo to the machine that opened it.Selecting one manager does not import or probe the other. Pixi rejects Python interpreter overrides, which its script environments cannot honor, and falls back cleanly for app-host notebooks without script metadata. CLI, adapter, and fixture tests cover both managers; the guide documents the remaining Pixi limitations.