Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion src/plugins/claude/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
#
# The Claude Code marketplace consumes a repo whose ROOT is the marketplace:
# .claude-plugin/marketplace.json marketplace manifest
# plugins/braintrust/ skills plugin (MCP + skills)
# plugins/trace-claude-code/ tracing plugin (thin daemon hook shim)
#
# Everything is plain shell + config — no compiled artifacts — so the whole
Expand Down
8 changes: 1 addition & 7 deletions src/plugins/claude/content/.claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,12 @@
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
"name": "braintrust-claude-plugin",
"version": "1.3.1",
"description": "Braintrust plugins for LLM evaluation, logging, and observability",
"description": "Braintrust session tracing for Claude Code",
"owner": {
"name": "Braintrust",
"email": "support@braintrust.dev"
},
"plugins": [
{
"name": "braintrust",
"description": "Enables AI agents to use Braintrust for LLM evaluation, logging, and observability. Provides correct API usage, working examples, and helper scripts.",
"source": "./plugins/braintrust",
"category": "development"
},
{
"name": "trace-claude-code",
"description": "Automatically trace Claude Code conversations to Braintrust. Captures user messages, assistant responses, and tool calls for observability.",
Expand Down
7 changes: 2 additions & 5 deletions src/plugins/claude/content/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## About this repository

This is the **Braintrust Claude Code plugin marketplace** - a repository that distributes Claude Code plugins for Braintrust integration.
This is the **Braintrust Claude Code plugin marketplace** - a repository that distributes Claude Code tracing for Braintrust.

### Structure

Expand All @@ -11,23 +11,20 @@ claude-plugin/
├── .claude-plugin/
│ └── marketplace.json # Marketplace catalog (lists available plugins)
├── plugins/
│ ├── braintrust/ # Plugin: Braintrust evaluation & logging
│ └── trace-claude-code/ # Plugin: Session tracing to Braintrust
└── evals/ # Evaluation suite for testing the plugins
└── evals/ # Evaluation suite for Braintrust MCP behavior
```

### Plugins

| Plugin | Description |
|--------|-------------|
| `braintrust` | Enables AI agents to use Braintrust for LLM evaluation, logging, and observability. Includes MCP server config and the `troubleshoot-braintrust-mcp` skill. |
| `trace-claude-code` | Forwards Claude Code lifecycle hooks to `bt trace hook --source claude-code`; the shared daemon builds and delivers traces. |

### Terminology

- **Marketplace**: A repository with a `marketplace.json` that catalogs multiple plugins for distribution
- **Plugin**: An installable unit with its own `.claude-plugin/plugin.json` manifest
- **Skill**: A capability within a plugin (e.g., `troubleshoot-braintrust-mcp` is a skill in the `braintrust` plugin)

## Style conventions

Expand Down
7 changes: 3 additions & 4 deletions src/plugins/claude/content/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Developing the Claude plugins
# Developing the Claude plugin

This distribution tree is generated from the
`braintrustdata/braintrust-coding-agent-plugins` monorepo. Make source changes
Expand All @@ -9,7 +9,6 @@ there rather than editing the generated distribution repository.
Load a plugin directly from the assembled tree:

```bash
claude --plugin-dir /path/to/repo/src/plugins/claude/content/plugins/braintrust
claude --plugin-dir /path/to/repo/src/plugins/claude/content/plugins/trace-claude-code
```

Expand All @@ -29,7 +28,7 @@ cargo test --manifest-path bt-daemon/Cargo.toml --all-features --locked

## Evaluation suite

The `evals/` directory tests the Braintrust skills plugin:
The `evals/` directory tests Braintrust MCP behavior directly:

```bash
cd evals
Expand All @@ -38,6 +37,6 @@ uv run braintrust eval .

## Releases

Versions are stored in each plugin's `.claude-plugin/plugin.json`. Use the
The version is stored in the plugin's `.claude-plugin/plugin.json`. Use the
monorepo's release workflow to bump versions, publish the generated distribution
tree, tag the release, and create release notes.
29 changes: 8 additions & 21 deletions src/plugins/claude/content/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
> Don't edit files here — make changes and file issues in that repository, and they
> will be rebuilt into this one.

A Claude Code plugin marketplace for [Braintrust](https://braintrust.dev) integration - LLM evaluation, logging, observability, and session tracing.
A Claude Code plugin marketplace for tracing Claude Code sessions to [Braintrust](https://braintrust.dev).

## Prerequisites

Expand All @@ -14,14 +14,13 @@ A Claude Code plugin marketplace for [Braintrust](https://braintrust.dev) integr

## Supported Claude surfaces

These marketplace plugins support Claude Code CLI and Claude Code mode in the
desktop app. They do not currently support the Cowork tab, which runs tools and
This marketplace plugin supports Claude Code CLI and Claude Code mode in the
desktop app. It does not currently support the Cowork tab, which runs tools and
hooks inside a separate VM without the host's `bt` installation, Braintrust
configuration, or environment variables.

In Cowork, use the Braintrust connector provided through Claude for MCP access.
The connector is separate from the `braintrust` marketplace plugin. Automatic
Cowork session tracing is not currently supported.
Automatic Cowork session tracing is not currently supported.

## Installation

Expand All @@ -31,22 +30,7 @@ Add the marketplace:
claude plugin marketplace add braintrustdata/braintrust-claude-plugin
```

Then install the plugins you need:

## Plugins

### braintrust

Enables AI agents to use Braintrust for LLM evaluation, logging, and observability.

- Query Braintrust projects, experiments, datasets, and logs
- Instrument your code with the Braintrust SDK and write evals

```bash
claude plugin install braintrust@braintrust-claude-plugin
```

### trace-claude-code
Then enable tracing:

Automatically traces Claude Code conversations to Braintrust through the shared
Braintrust daemon. The plugin contains only a fail-open hook forwarder; `bt`
Expand All @@ -63,6 +47,9 @@ Every registered lifecycle event is forwarded synchronously to
`bt trace hook --source claude-code`, preserving per-session ordering. Hook
failures never fail a Claude Code turn.

This marketplace does not install or configure the Braintrust MCP server. Use
your agent's native connector or MCP configuration when you want MCP access.

#### Additional root metadata

For a persistent route, pass a JSON object to `bt trace enable claude
Expand Down

This file was deleted.

11 changes: 0 additions & 11 deletions src/plugins/claude/content/plugins/braintrust/.mcp.json

This file was deleted.

This file was deleted.

24 changes: 20 additions & 4 deletions src/plugins/claude/validate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# validate.sh — Sanity-check a built Claude Code dist tree in $1 before publishing.
#
# Fails (non-zero) on the mistakes that would ship a broken marketplace:
# - missing marketplace manifest / plugin manifests / hooks / skill
# - missing marketplace manifest / tracing plugin manifest / hooks
# - malformed JSON in any manifest
# - marketplace entries whose `source` path does not exist in the tree
#
Expand All @@ -30,9 +30,6 @@ check_json "$MARKETPLACE"

# Required files for each shipped plugin.
required=(
"plugins/braintrust/.claude-plugin/plugin.json"
"plugins/braintrust/.mcp.json"
"plugins/braintrust/skills/troubleshoot-braintrust-mcp/SKILL.md"
"plugins/trace-claude-code/.claude-plugin/plugin.json"
"plugins/trace-claude-code/hooks/hooks.json"
"plugins/trace-claude-code/hooks/forward.sh"
Expand All @@ -42,6 +39,25 @@ for rel in "${required[@]}"; do
case "$rel" in *.json) check_json "$TARGET_DIR/$rel";; esac
done

python3 - "$MARKETPLACE" "$TARGET_DIR/plugins" <<'PY' \
|| fail "Claude marketplace must contain only the tracing plugin"
import json
import sys
from pathlib import Path

with open(sys.argv[1]) as f:
plugins = json.load(f)["plugins"]

assert [plugin["name"] for plugin in plugins] == ["trace-claude-code"]
assert sorted(path.name for path in Path(sys.argv[2]).iterdir() if path.is_dir()) == [
"trace-claude-code"
]
PY

if find "$TARGET_DIR" -name '.mcp.json' -print -quit | grep -q .; then
fail "Claude dist still contains an MCP proxy configuration"
fi

python3 - "$TARGET_DIR/plugins/trace-claude-code/hooks/hooks.json" <<'PY' \
|| fail "Claude hooks do not all use the blocking daemon forwarder"
import json
Expand Down
1 change: 0 additions & 1 deletion src/plugins/codex/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
#
# The Codex marketplace consumes a repo whose ROOT is the marketplace:
# .agents/plugins/marketplace.json marketplace manifest
# plugins/braintrust-codex-plugin/ skills plugin (MCP + skills)
# plugins/trace-codex/ tracing plugin (thin daemon hook shims)
#
# The tracing plugin contains no tracing runtime or platform-specific binary;
Expand Down
14 changes: 1 addition & 13 deletions src/plugins/codex/content/.agents/plugins/marketplace.json
Original file line number Diff line number Diff line change
@@ -1,21 +1,9 @@
{
"name": "braintrust-codex-plugins",
"interface": {
"displayName": "Braintrust Codex Plugins"
"displayName": "Braintrust Codex Plugin"
},
"plugins": [
{
"name": "braintrust",
"source": {
"source": "local",
"path": "./plugins/braintrust-codex-plugin"
},
"policy": {
"installation": "AVAILABLE",
"authentication": "ON_INSTALL"
},
"category": "Developer tools"
},
{
"name": "trace-codex",
"source": {
Expand Down
19 changes: 5 additions & 14 deletions src/plugins/codex/content/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,11 @@ Guidelines for AI agents working in this repo.

## Repo purpose

This repo is a monorepo of two independent [Codex marketplace plugins](https://developers.openai.com/codex/plugins):
This repo distributes one [Codex marketplace plugin](https://developers.openai.com/codex/plugins):

- `plugins/braintrust-codex-plugin/` — packages the [Braintrust MCP server](https://www.braintrust.dev/docs/integrations/developer-tools/mcp) plus a routing skill.
- `plugins/trace-codex/` — an opt-in plugin that traces Codex sessions to Braintrust (session, turn, and tool spans) via Codex lifecycle hooks. Do **not** merge tracing behavior into the MCP/skills plugin; they are separate, independently installable plugins.
- `plugins/trace-codex/` — an opt-in plugin that traces Codex sessions to Braintrust (session, turn, and tool spans) via Codex lifecycle hooks.

Both plugins are listed as separate entries in `.agents/plugins/marketplace.json` (the repo marketplace).

Key files for the MCP/skills plugin:

- `plugins/braintrust-codex-plugin/.codex-plugin/plugin.json` — plugin manifest (version, UI metadata, default prompts)
- `plugins/braintrust-codex-plugin/.mcp.json` — MCP server definition
- `plugins/braintrust-codex-plugin/skills/braintrust/` — agent skills exposed through the plugin
The plugin is listed in `.agents/plugins/marketplace.json` (the repo marketplace).

Key files for the tracing plugin (see [`plugins/trace-codex/AGENTS.md`](plugins/trace-codex/AGENTS.md) for its architecture):

Expand All @@ -26,10 +19,8 @@ Key files for the tracing plugin (see [`plugins/trace-codex/AGENTS.md`](plugins/

## Making changes

- **Skills**: There is only one simple skill in this repo which handles routing and tool definitions, it should not be modified significantly.
- **MCP config**: edit `plugins/braintrust-codex-plugin/.mcp.json` to change the MCP server command or environment variables.
- **Plugin metadata**: edit the relevant `.codex-plugin/plugin.json` for display name, description, brand color, default prompts, etc.
- **Marketplace**: edit `.agents/plugins/marketplace.json` to change plugin entries, categories, or install policies.
- **Plugin metadata**: edit `plugins/trace-codex/.codex-plugin/plugin.json`.
- **Marketplace**: edit `.agents/plugins/marketplace.json` to change the plugin entry, category, or install policy.

## Releasing a new version

Expand Down
12 changes: 4 additions & 8 deletions src/plugins/codex/content/README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,19 @@
# Braintrust Codex Plugins
# Braintrust Codex Plugin

> **This repository is generated.** It is built from
> [braintrustdata/braintrust-coding-agent-plugins](https://github.com/braintrustdata/braintrust-coding-agent-plugins).
> Don't edit files here — make changes and file issues in that repository, and they
> will be rebuilt into this one.

Braintrust [Codex plugins](https://developers.openai.com/codex/plugins) — skills and daemon-backed session tracing.
Daemon-backed Braintrust session tracing for [Codex](https://developers.openai.com/codex/plugins).

## Quickstart

Add this repo as a Codex plugin marketplace:

```bash
codex plugin marketplace add braintrustdata/braintrust-codex-plugin
# OPTIONAL: TRACE CODEX PLUGIN
codex plugin add trace-codex@braintrust-codex-plugins
# OPTIONAL: SKILLS PLUGIN
codex plugin add braintrust@braintrust-codex-plugins
```

The recommended tracing setup is:
Expand All @@ -33,6 +30,5 @@ daemon. Restart Codex after setup.

See the plugin's [README](/plugins/trace-codex/README.md) for details.

## skills plugin

see the plugin's [README](/plugins/braintrust-codex-plugin/README.md) for details
This marketplace does not install or configure the Braintrust MCP server. Use
Codex's native connector or MCP configuration when you want MCP access.
Loading
Loading