Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
9e9490b
feat: add Factory Droid client integration
shawn-kim-ai Sep 19, 2026
ed4bdc2
fix: address Droid integration review feedback (#5193)
shawn-kim-ai Sep 20, 2026
69c5ce8
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
bf78263
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
10f599c
fix(droid): resolve review readiness findings
shawn-kim-ai Sep 20, 2026
6a2c657
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
b153329
docs: refresh Droid integration screenshot
shawn-kim-ai Sep 20, 2026
cb4fb43
fix(droid): reject credential-bearing verification exports
shawn-kim-ai Sep 20, 2026
5de4904
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
b9a0f30
test: bound standalone shim probe
shawn-kim-ai Sep 20, 2026
ea88f3c
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
c2f3b86
Address PR review feedback (#5193)
shawn-kim-ai Sep 20, 2026
dac9977
Test Droid unknown image modalities (#5193)
shawn-kim-ai Sep 20, 2026
c99e7c4
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
c555c44
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
324e408
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
71b2b50
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
1c52daf
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
d0f78ce
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
358efa9
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
2c4cadd
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
408b5c7
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
4bac816
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
338dc38
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
483213f
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
7857d9e
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
cb96120
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
ae7962a
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
26ca115
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 20, 2026
6e64779
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
8c7cd06
fix(gui): accept Droid mutation plan paths
shawn-kim-ai Sep 21, 2026
588488a
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
96e1c6d
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
921cbc0
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
4336c09
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
bb7076c
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
e9d2447
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
525938a
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
1d02ac6
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
9d5691c
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
de10be5
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
0bd5c3d
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
2b85470
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
c0a83b2
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
fa7f2a6
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
a65b7b5
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
a95f9c4
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
0589fc3
test(desktop): follow moved ownership validator
shawn-kim-ai Sep 21, 2026
ad6c0a8
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
1af3370
Merge remote-tracking branch 'origin/dev' into codex/factory-droid-in…
shawn-kim-ai Sep 21, 2026
0282d93
fix(gui): clarify Factory Droid undo wording (#5193)
shawn-kim-ai Sep 21, 2026
43a05d9
Merge current dev into Factory Droid integration PR
shawn-kim-ai Sep 25, 2026
7255a44
test: isolate readiness checks from host state and DNS
shawn-kim-ai Sep 25, 2026
268f14b
test: keep provider validation within file-size cap
shawn-kim-ai Sep 25, 2026
aa8462e
Merge latest dev and retain heartbeat parity coverage
shawn-kim-ai Sep 25, 2026
f5925ac
Merge dev and preserve Droid integration documentation
shawn-kim-ai Sep 26, 2026
3607336
docs: align translated integration refresh rosters
shawn-kim-ai Sep 26, 2026
8ae73ae
test: isolate service claim refusal from legacy state
shawn-kim-ai Sep 26, 2026
68aad08
test: run history lock checks in a fresh process
shawn-kim-ai Sep 26, 2026
91090f8
Merge dev for Droid review readiness
shawn-kim-ai Sep 26, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions .agents/skills/verify-droid-integration/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
name: verify-droid-integration
description: Verify the Factory Droid CLI outbound integration against a live OpenCodex proxy, including every active model, streaming, reasoning, tools, images, and long context.
---

# Verify Factory Droid integration

Use this skill after changing Droid export, client integration ownership, model metadata, or the
OpenAI-compatible chat path. It drives the real `droid exec` CLI with a process-only settings file.
It never writes `~/.factory/settings.json`.

## Launch

Use the user's existing signed-in OpenCodex instance when `curl -fsS
http://127.0.0.1:10100/healthz` succeeds. The provider credentials live in that instance's
`OPENCODEX_HOME`, so do not start a second proxy against the same home.

If no proxy is running, start this checkout and keep its PID:

```bash
bun run src/cli/index.ts start --port 10100
```

Wait for `http://127.0.0.1:10100/healthz`. Stop only the PID you started.

## Doctor

Run the no-inference pass first:

```bash
bun .agents/skills/verify-droid-integration/scripts/run.ts --dry-run
```

It exports the live active catalog through `ocx export --client droid`, checks proxy health and
Droid version, validates unique generic-provider rows, and asks Droid to resolve every custom model
ID with `--list-tools`. A failure makes the instance unfit for the paid live pass.

## Drive

Run the full matrix:

```bash
bun .agents/skills/verify-droid-integration/scripts/run.ts \
--concurrency 3 \
--timeout-ms 180000 \
--long-words 20000
```

The active model set is frozen from the exported config at run start. Each model gets catalog,
normal text, stream JSON, reasoning when advertised, Read tool call/result, image Read/OCR when
advertised, and long-context coverage. Use `--models <csv>` or `--cases <csv>` only for diagnosis;
the PR proof is the unfiltered run.

## Evidence

Each run writes `.tmp/verify-droid-integration/<timestamp>/`. The directory contains the generated
settings, export and health evidence, one command/result JSON per model and case, `summary.json`,
and `failures.json`. The command, exit code, stdout, stderr, timeout state, and duration preserve
both the action and result. Keep the full directory through review.

Passing means no `fail` or `error` result. `fail` records a completed assertion mismatch; `error`
records a timeout, nonzero exit, or caught execution exception. `unsupported` is accepted only when
the exported model lacks the corresponding reasoning or image capability. The generated settings
must remain free of real keys.

## Cleanup

The helper starts no proxy and changes no persistent Factory settings. Delete diagnostic prompt and
marker files only by deleting a run directory after its evidence is no longer needed. Do not remove
the proof directory during verification. If this skill started a proxy, stop that exact PID after
the final drive. Droid records its own exec sessions in the signed-in user's normal history; the
helper records their IDs in command output and does not delete unrelated history.

## Helpers

`scripts/run.ts` is executable and self-documenting:

```bash
bun .agents/skills/verify-droid-integration/scripts/run.ts --help
```

## Failure recovery

Read `failures.json`, rerun only the affected model and case, and preserve the new run beside the
original. After a timeout or transport failure, repeat the Doctor pass before another paid request.
If health is green but Droid remains wedged, end that `droid exec` process and start a fresh run;
never reuse a failed session.

## Feature map

Read [features/README.md](features/README.md), then maintain it with
`/maintain-verification-skill` after user-facing integration changes.
12 changes: 12 additions & 0 deletions .agents/skills/verify-droid-integration/features/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Factory Droid verification feature map

| Feature | Proof |
| --- | --- |
| [Catalog export](catalog-export.md) | Active models become stable, keyless Factory custom rows |
| [Text, stream, and reasoning](text-stream-reasoning.md) | Droid completes normal and stream JSON requests and applies advertised effort |
| [Tool round trip](tool-roundtrip.md) | The model emits a Read call, receives the result, and uses it |
| [Image input](image-input.md) | An image-capable model reads pixels through Droid and returns the expected headline |
| [Long context](long-context.md) | Both boundary markers survive a deterministic long prompt |

The full verifier drives every exported active model. A filtered rerun is diagnostic evidence, not
a replacement for the complete matrix.
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Catalog export

## Sub-features

- Live active-model filtering
- Stable `custom:opencodex:<selector>` IDs
- Exact generic Chat provider and loopback URL
- A 16,384-token response ceiling instead of the full context window
- Context, image, and reasoning metadata
- Keyless config and per-row ownership

## How to get to it (user POV)

Open **Integrations → Factory Droid**, or run `ocx integration client enable --client droid`.
For a non-persistent preview, run `ocx export --client droid --out <temp> --force`.

## Driving it with Droid CLI

Run the verifier with `--dry-run`. Inspect `settings.json` and each model's `catalog.json`.

## Gotchas

The export includes hub-approved Fast rows as active selectors. Do not compare it with a raw provider
count. The inbound Factory bridge in the docs is a separate direction.
22 changes: 22 additions & 0 deletions .agents/skills/verify-droid-integration/features/image-input.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Image input

## Sub-features

- Catalog image capability maps to `noImageSupport: false`
- Droid Read loads the image
- The routed model reads visible pixels

## How to get to it (user POV)

Select an image-capable OpenCodex custom model and ask Droid to inspect an image file.

## Driving it with Droid CLI

Run `--cases image`. The helper copies `assets/pr-gate-screenshot-required.png` to an opaque random
filename, asks Droid to read that copy without putting the headline in the prompt, then requires the
exact pixel-only answer plus a matching Read call and two-part text/image result payload.

## Gotchas

Models exported with `noImageSupport: true` are `unsupported`. Filename or alt-text inference is not
enough; the expected headline exists only in the image pixels.
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Long context

## Sub-features

- Prompt-file ingestion
- Deterministic context padding
- Recall of markers at both boundaries

## How to get to it (user POV)

Pass a long prompt file to a selected OpenCodex custom model with `droid exec --file <path>`.

## Driving it with Droid CLI

Run `--cases long-context --long-words 20000`. The helper generates model-specific start and end
markers and requires both in the exact final response.

## Gotchas

`--long-words` is a deterministic stress size, not an exact tokenizer count. A timeout is a failure,
not an unsupported result; rerun after Doctor before attributing it to the model.
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Text, stream, and reasoning

## Sub-features

- Normal JSON completion
- Stream JSON lifecycle with a final completion
- Advertised reasoning-effort selection

## How to get to it (user POV)

Select an `OpenCodex: ...` custom model in Droid, then run a prompt normally or pass
`--output-format stream-json` and `--reasoning-effort <level>` to `droid exec`.

## Driving it with Droid CLI

Run the full verifier or `--cases text,stream,reasoning`. Proof requires exact marker text, a stream
`system` event, a `completion` event, and the requested effort in the init event.

## Gotchas

A model with no exported effort ladder is `unsupported` for the reasoning case. A successful final
line alone does not prove the stream lifecycle.
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Tool round trip

## Sub-features

- Read tool definition reaches the model
- Tool call arguments select the marker file
- Droid returns a successful tool result
- The final answer includes content from that result

## How to get to it (user POV)

Run Droid in a repository and ask the selected OpenCodex model to read a file.

## Driving it with Droid CLI

Run `--cases tool`. The helper creates a unique marker file inside the evidence directory and
requires a Read call for that exact path, its matching successful result containing the marker,
and an exact final completion.

## Gotchas

Prompt compliance without a tool event is a failure. The verifier limits available tools to Read so
the proof remains read-only.
Loading
Loading