feat(client-generator): agent-friendly generators — python, go, php, cli + neutral authoring toolkit and eject workflow - #3016
feat(client-generator): agent-friendly generators — python, go, php, cli + neutral authoring toolkit and eject workflow#3016Marshevskyy wants to merge 181 commits into
Conversation
🦋 Changeset detectedLatest commit: 7ed38b7 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Performance Benchmark (Lower is Faster)
|
|
📦 A new experimental 🧪 version v0.0.0-snapshot.1785856098 of Redocly CLI has been published for testing. Install with NPM: npm install @redocly/cli@0.0.0-snapshot.1785856098 |
Feedback from Rebilly (Replay Admin + Rebilly Core)Tried
Overall: generation succeeds for both; TypeScript still works when co-selected with 1. Core list one-shot vs
|
Follow-up: Python generator (same snapshot)Tried
Overall: generation succeeds; sync + async clients work; offline smokes pass for Admin cursor pagination and Core offset pagination + Python is in good shape relative to PHP on several points we flagged earlier — calling those out as already-good, then listing Python-specific asks. Already better than PHP (positive)
Python-specific asks1. Output filename is not a normal import pathEmitted names follow the TS stem:
Neither is a valid/idiomatic Python module name ( Ask: emit an importable name by default, e.g. 2. Cross-language auth config key mismatchPython resolves API keys from Ask: document the per-language auth dict shape next to each other (or normalize to one key with a documented alias). Easy footgun when someone copies a TS/PHP snippet into Python. 3. Reserved-word fields (
|
Clarification on single-file outputOne more product note on the earlier “large single-file” asks (PHP ~4.7MB / Python ~4.4MB for Rebilly Core): We like the single-file default and would keep it. It’s a great “Download client” artifact from Redoc/API docs — one Please treat the size/split comments as optional escape hatches for huge descriptions, not a request to change the default:
The other asks still stand, especially for Python: importable filename ( |
Follow-up: Go generator (same snapshot)Tried
Overall: generation succeeds; Same single-file preference as before: keep the one-file default for Redoc/API docs downloads. Core Go is notably smaller than Core PHP (~4.7MB) / Python (~4.4MB), which helps that story. Already in good shape (positive)
Go-specific asks1. Emit
|
Follow-up:
|
|
|
||
| Added agent-friendly client generation: built-in `python`, `go`, `php`, and `cli` generators, a language-neutral authoring toolkit with a per-generator `AGENTS.md` skill, an `eject-generator` command, `x-codeSamples` output, and verification against large real-world descriptions — with every generator now emitting through source-text templates. | ||
|
|
||
| **Note:** the AST exports (`ts`, `printStatements`, `schemaToTypeNode`, …) were removed from `@redocly/client-generator/generate` in favor of the text toolkit (`tsType`, `tsJsdoc`, `codeLiteral`). |
| - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0 | ||
| with: | ||
| python-version: '3.12' | ||
| - name: Install httpx (the large-descriptions Python import bar needs it) | ||
| run: pip install httpx | ||
| - name: Cache the pinned GitHub REST description | ||
| uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4 | ||
| with: | ||
| path: tests/e2e/generate-client/.cache | ||
| key: large-descriptions-${{ hashFiles('tests/e2e/generate-client/large-descriptions.test.ts') }} |
There was a problem hiding this comment.
I think we need a separate test suite for generator.
I think it will be very big and slow eventually when we start testing compiled languages.
There was a problem hiding this comment.
Agreed, and it's already the biggest suite. Splitting generator tests (client-generator unit +
tests/e2e/generate-client, including the large-description compile bars) into their own vitest
suite and CI job so compiled-language testing can grow without slowing the main e2e job.
| Your agent (or you) edits the generator, `redocly generate-client` rebuilds the client, and next week's spec change regenerates with the customization intact. | ||
|
|
||
| Ejectable generators: `python`, `go`, `php` — the language generators built on the language-neutral authoring toolkit. | ||
| The TypeScript `sdk` and its satellite generators are customized through `client.setup`, middleware, and configuration instead; running `eject-generator sdk` prints that guidance. |
There was a problem hiding this comment.
I think typescript and satellites should be ejectable too as separate generators
There was a problem hiding this comment.
Agreed. Docs now say every built-in is ejectable. Mechanically the language generators are one
self-contained file, while the TypeScript ones are thin entries over shared emitters, so eject
will bundle each generator with the emitters it uses into a single .mjs, keeping the
@redocly/client-generator imports external. sdk.mjs will be large, but it's one file you own,
which is the point.
| | ---------- | ------- | ---------------------------------------------------------------------------------------------------- | | ||
| | generator | string | Built-in generator to eject: `python`, `go`, or `php`. | | ||
| | `--dir` | string | Directory to eject into. Default `./generators`. | | ||
| | `--update` | boolean | Three-way merge a newer generator version into your customized copy; conflicts get standard markers. | |
| Ejecting writes four things: | ||
|
|
||
| - `<dir>/<name>.mjs` — the generator, the exact code the built-in runs, readable plain ESM. | ||
| - `<dir>/.pristine/<name>.mjs` — a pristine snapshot (commit it); `--update` uses it as the merge base. |
There was a problem hiding this comment.
I think we don't need it. can update compare with the latest upstream version?
There was a problem hiding this comment.
Dropping .pristine/ entirely. --update now uses the version recorded in the ejected file's
header as the merge base, so nothing extra is committed and there's nothing to keep in sync.
| ### Code samples for docs | ||
|
|
||
| A generator that knows how to call an operation can also document it: implement the optional `sample(operation, ctx)` hook to return one idiomatic snippet (`{ lang, label, source }`) per operation. | ||
| With `codeSamples: true` in the `client` block, generation collects every selected generator's samples into `<output stem>.code-samples.yaml` — an [OpenAPI Overlay](https://spec.openapis.org/overlay/latest.html) adding `x-codeSamples` per operation, ready for docs tooling to apply. |
There was a problem hiding this comment.
I think we don't have support for Overlays. We need to either add support for overlays in bundle or figure out some other format
|
|
||
| The `cli` generator emits `<stem>.cli.ts` — a zero-dependency, bin-ready command-line interface over the generated client. | ||
| Path params are positional, query params become typed `--kebab-name` flags (enums list their choices in `--help`, array params repeat the flag), and JSON request bodies arrive via `--json '<json>'`, `--json @file.json`, or `--json @-` (stdin). | ||
| When `zod` is co-selected, requests are validated before they are sent. |
There was a problem hiding this comment.
I think it shoud just do it by default without the need to coselect zod
There was a problem hiding this comment.
Done. cli declares requires: ['sdk', 'zod'] and the resolver now pulls prerequisites in, so
--generator cli alone emits all three files and validates (exit 3) with nothing extra to
select. Same for the wrappers: --generator tanstack-query brings the sdk it wraps. The
trade-off is a zod runtime dependency for the CLI
| npx tsx src/client.cli.ts orders listOrders --status open --limit 10 | ||
| npx tsx src/client.cli.ts orders createOrder --json @order.json | ||
| npx tsx src/client.cli.ts orders listOrders --page-all # one JSON page per line | ||
| npx tsx src/client.cli.ts schema createOrder # request/response schemas |
| `tanstack-query`, `swr`, and `cli` wrap the throw-mode `sdk` client, so they require `--error-mode throw`; `transformers` requires `--date-type Date`. | ||
| See the [`zod`](https://github.com/Redocly/redocly-cli/tree/main/tests/e2e/generate-client/examples/zod), [`tanstack-query`](https://github.com/Redocly/redocly-cli/tree/main/tests/e2e/generate-client/examples/tanstack-query), and [`mock`](https://github.com/Redocly/redocly-cli/tree/main/tests/e2e/generate-client/examples/mock) examples. | ||
|
|
||
| ### Generated CLI |
There was a problem hiding this comment.
I think it should also be able to generate documentation for the CLI (markdown file)
But the same applies for other clients so it can be the next stage.
There was a problem hiding this comment.
It may need some documentation templates that can be ejected then too.
There was a problem hiding this comment.
Built it — there's now a cli-docs generator that writes .cli.md beside the generated CLI: usage, global flags, credential environment variables, the exit-code table, and one section per command with its positional arguments and flags (type, required, enum choices, description). --generator cli-docs is the whole selection, since it pulls in the CLI it documents.
On ejectable templates: the renderer is the template. Every built-in generator is ejectable now, so redocly eject-generator cli-docs hands you the page layout as code you own — same language and toolkit as everything else, no template syntax and no extra dependency to learn. Light customization stays declarative through the generator's own options (title, frontmatter) under client.options.cli-docs; anything structural is an eject. We deliberately didn't add a template engine, because that would be a second customization mechanism sitting next to eject.
One property worth calling out: the page renders from the same command table the CLI dispatches on, and the same functions the runtime uses to address groups and name credential variables — so the docs can't drift from the tool. An e2e walks the generated CLI's own --help at both levels and fails if any command it lists is missing a section.
| The `python` generator emits a self-contained `<stem>.py` next to the configured output — a full Python SDK over [httpx](https://www.python-httpx.org/) (`pip install httpx`, Python ≥ 3.9): | ||
| typed dataclass models (allOf flattened, enums, discriminated unions decoded by their discriminator), a `Client` and an `AsyncClient` with one method per operation, auth, retries with `Retry-After` and jittered backoff, timeouts, idempotency keys, middleware hooks, pagination iterators (`<op>_pages()` / `<op>_items()`, `async for` variants), SSE streaming, multipart bodies, `<op>_with_headers()` envelope variants for operations that declare response headers, and a `Servers` class for templated server URLs. |
There was a problem hiding this comment.
Why we need all of this details. It should be same as typescript one
There was a problem hiding this comment.
it doesn't support any of options we support for typescript? why?
There was a problem hiding this comment.
You were right on both counts, and one of them was a bug: serverUrl was silently ignored by the
language generators (fixed, 795f58ca9). dateType: Date is now implemented for all three
(ea3795e47). Options a language genuinely can't apply no longer vanish — they warn with the
reason, or fail fast where the output would be wrong (d6cf2ba4e). The docs no longer enumerate
per-language caveats: the language sections state that these are the TypeScript client in another
language, with one table for the differences the language forces (error idiom, date type, header
envelope, auth shape, reserved-word suffix, file layout).
| A generator adds artifacts _next to_ the client — it doesn't change the generated client's behavior; for that, use [publisher defaults](#publisher-defaults) or let the consumer compose [middleware](./use-generated-client.md#middleware). | ||
|
|
||
| A generator is `{ name, run }` (plus optional compatibility metadata); author it with `defineGenerator` from the package root, and build real TypeScript with the emit toolkit from `@redocly/client-generator/generate` — the same `ts.factory` + printer the built-in generators use, so the schema→type mapping matches the sdk's exactly: | ||
| A generator is `{ name, run }` (plus optional compatibility metadata); author it with `defineGenerator` from the package root. |
There was a problem hiding this comment.
maybe it should also export additional options this generator may support? (e.g. as json schema)
We can support them via config only for example.
There was a problem hiding this comment.
Yes, a generator declares options as a JSON Schema, validated before run, and publishers set them under client.options.<generator> in config.
…, discriminatorCases, nullability, enums)
… from the root and /generate
…oaded built-in generators
… hooks, sdk as reference implementation
…d test, and guide rework
…lkit-import names
… with build-time embed
… table with reflective hydration
…mples and a dogfooding guard
… helpers — slice-2 lessons landed, python items typed
…tdlib-only, vetted, embedded
vadyvas
added
snapshot
|
📦 A new experimental 🧪 version v0.0.0-snapshot.1786626470 of Redocly CLI has been published for testing. Install with NPM: npm install @redocly/cli@0.0.0-snapshot.1786626470 |
| ```sh | ||
| npm run generate | ||
|
|
||
| npx tsx src/api/client.cli.ts --help |
There was a problem hiding this comment.
Why do we need to use npx/tsx? Cannot we just execute the script with node?
There was a problem hiding this comment.
Plain node works, with two qualifiers, which is why the example shows tsx:
- Node runs TypeScript natively only on newer lines (unflagged since 22.18/23.6, --experimental-strip-types before that, not at all on our 20.19 floor) — tsx works on every Node we support.
- More binding: with the default importExt: js, the generated CLI imports its sibling as ./client.js, and Node's literal resolver won't map that to client.ts — tsx does. Generate with --import-ext ts and plain node src/api/client.cli.ts works — that's the documented "Run with Node directly" path, the node-native example, and the e2e even runs the CLI under --experimental-strip-types to pin that it stays erasable.
So: tsx = default config on any supported Node; plain node = importExt: ts on Node ≥ 22.18. Happy to add a one-line pointer to the node-native run in this README.
| ``` | ||
|
|
||
| `binName` sets the name in the help output and the prefix of the credential variables. | ||
| It does not install a `cafe` executable. |
There was a problem hiding this comment.
So why then we call it a bin name?
There was a problem hiding this comment.
Because it's the name of the executable this module presents itself as — the $0 in every usage line, the base the credential variables derive from (CAFE_TOKEN), and the exact key you'd put under package.json bin when you ship it. "bin" is the ecosystem's word for that slot, so the option names where the value ultimately lands. What generation can't do is the installing itself — wiring bin and compiling is the consuming package's build — which is all that sentence is drawing a line around. Open to wording the doc line better if it reads as an apology rather than a boundary.
Co-authored-by: Jacek Łękawa <164185257+JLekawa@users.noreply.github.com>
Co-authored-by: Jacek Łękawa <164185257+JLekawa@users.noreply.github.com>
Co-authored-by: Jacek Łękawa <164185257+JLekawa@users.noreply.github.com>
Co-authored-by: Jacek Łękawa <164185257+JLekawa@users.noreply.github.com>
Co-authored-by: Jacek Łękawa <164185257+JLekawa@users.noreply.github.com>
Co-authored-by: Jacek Łękawa <164185257+JLekawa@users.noreply.github.com>
Co-authored-by: Jacek Łękawa <164185257+JLekawa@users.noreply.github.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 5 total unresolved issues (including 4 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 15b38e8. Configure here.
| const fits = | ||
| convention.style === 'link' || | ||
| (typeof advance === 'string' && op.queryParams.some((param) => param.name === advance)); | ||
| if (fits) rule = convention as Record<string, unknown>; |
There was a problem hiding this comment.
Link pagination fits every operation
High Severity
paginationRuleFor treats a link convention as fitting every operation because that style has no advance parameter. The TypeScript emitter still requires a documented Link header before applying the convention, and the docs describe that same rule. Python, Go, and PHP call this helper directly, so a client.pagination.style: link config attaches page iterators to operations that are not paginated.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 15b38e8. Configure here.
There was a problem hiding this comment.
Confirmed and fixed. The neutral paginationRuleFor treated a link convention as fitting every operation, while the TypeScript emitter (and the docs) gate the convention on a documented Link response header — so client.pagination.style: link gave the Python/Go/PHP SDKs page iterators on operations that don't paginate. The helper now applies the same gate the TS emitter uses (successResponseHeaders containing link); an explicit per-operation rule or x-redoclyPagination extension still applies regardless, mirroring the TS explicit-rule path. Covered by a unit test with both the undocumented and documented cases. Since ejected language generators call this helper from the installed package, the fix reaches them by npm update — no re-eject needed.
9 participants
What this adds
Client generation for languages beyond TypeScript, built so that both we and our users extend it with AI agents driven by per-generator design skills.
New built-in generators
python— self-contained client over httpx: typed dataclasses, sync + async clients, pagination iterators, SSE, multipart, retries, discriminated-union decode via aDISCRIMINATORSregistry.go— stdlib-only single file: structs with json tags, typed-const enums,(T, error)methods,context.Context, range-over-func pagination, SSE.goPackagesets the package clause.php— PHP ≥ 8.1 over the curl extension, zero Composer dependencies: promoted-constructor models, native enums, named-argument methods,Generator-based pagination, SSE over a curl_multi pump.cli— a bin-ready command-line interface over the sdk. It validates requests by default: selectingclipulls in the generators it needs (sdkandzod), so nothing extra has to be listed.binNamenames the command.cli-docs— the Markdown reference for that CLI (<stem>.cli.md): usage, global flags, credential variables, exit codes, and every command with its arguments and flags. It renders from the same command table the CLI dispatches on, so the page cannot drift from the tool.All three SDK languages have parity on auth (bearer/basic/apiKey with token providers), retries with
Retry-After+ jittered backoff, timeouts, idempotency keys, middleware, pagination, SSE, multipart, binary downloads, templated-server helpers (Serversclass /<Name>URLfunctions), and response-header envelopes (<op>WithHeadersvariants mirroring the TS{ envelope: true }option from #3002).The authoring model
Printer(indentation-aware source-text builder), naming (casing,identifierFor,RESERVED_WORDS), schema semantics (flattenAllOf,discriminatorCases, nullability,enumValues,headerCoerceType,schemaAtPointer),paginationRuleFor, andNotSupportedErrorfor rejecting an option the generator can't honor. A dogfooding guard test pins that the built-in language generators use nothing else.typescriptdependency in the authoring path. The TypeScript emitters render source text like everything else; thets.factoryexports are gone.typescriptremains an optional peer needed only to bake a--setupmodule, the one place we parse TypeScript.run(unknown key, wrong type, value outside anenum, missing required key) with defaults applied. Publishers set them underclient.options.<generator>.Eject workflow
redocly eject-generator <name>vendors any built-in generator into the repo as an editable.mjs. A language generator ships as its own source; a TypeScript generator ships bundled with the emitters it uses (unminified, one comment per source module), sosdk,zod,mock,swr,tanstack-query,transformers,cli, andcli-docsare ejectable too. An ejected-unmodified generator produces byte-identical output, proven in e2e.Eject also:
.claude/skills/<name>-generator/SKILL.md) plus the shared authoring skill, where agents auto-load them, and leaves a short pointer beside the code;@redocly/client-generatorindevDependenciesand adds the entry toclient.generators, editing the config text so comments survive;--updatethree-way merges with no committed snapshot: the merge base is the version recorded in the ejected file's own header, fetched from the registry when it differs from the installed one.Compatibility
Generator compatibility is the package version under semver, not an invented number: a generator declares
requiresGenerator(^1.2.0,~1.2.0,>=1.2.0, or an exact version), and a CLI outside that range says which version it ships, which the generator needs, and how to reconcile them. Ejected generators record it automatically. Custom generators also run behind a validated contract: load-time shape validation, output-path containment (no writes outside--output), andrun()result validation. An IR-shape snapshot test forces the "additive or breaking?" question on any model change. Failures are attributable (Generator "<name>" failed: …) and categorized in telemetry along with eject/update outcomes (coarse categories only — never file contents, paths, or user-chosen names).Verification
tsc/py_compile+import /go build+go vet+gofmt/php -l+require(tests/e2e/generate-client/large-descriptions.test.ts).npm run generatorsruns the client-generator unit tests plustests/e2e/generate-client, sharded two ways, so a growing set of compiled-language bars cannot slow the job everything else shares.npm run e2ecovers everything else.tests/e2e/generate-client/examples/.Docs
New command page (
eject-generator), a customize-client-generation guide, per-language usage notes with one table for the differences each language forces, and configuration reference updates (goPackage,binName,options,codeSamples).Note for reviewers
The per-operation pagination extension is now
x-redoclyPagination(camelCase, like every other Redocly extension). A description that still declaresx-redocly-paginationsilently loses its pagination rule — called out in the changeset.Check yourself
Security
Note
High Risk
Large experimental surface (new generators, eject/update, CLI composition) plus breaking renames (
sdk→typescript, pagination extension) that can silently change behavior for existing OpenAPI descriptions and configs.Overview
Expands client generation beyond TypeScript with built-in
python,go,php,cli, andcli-docsgenerators, plus a language-neutral authoring toolkit (per-generator JSON Schema options,requiresGeneratorcompatibility) andredocly eject-generatorto vendor any built-in generator into the repo with agent skills and--updatethree-way merges.Breaking / config changes: the TypeScript generator is now
typescript(notsdk); pagination in specs must usex-redoclyPagination—x-redocly-paginationis no longer read. Newclientoptions includegoPackage,binName,cliOutput(multi-API CLI),codeSamples, andoptions.<generator>.Adds
npm run client-generatorsand a dedicated CI job (Python/Go/PHP compile bars, sharded) so generate-client e2e no longer shares the main e2e shard. CLI build copieseject-assets; telemetry gains coarsegenerate-client/eject-generatorcategories without paths or user-chosen names. Docs, CONTRIBUTING, and agent rules are updated for the new workflow.Reviewed by Cursor Bugbot for commit 7ed38b7. Bugbot is set up for automated code reviews on this repo. Configure here.