From 8dc21861af6d74141bef8948486edf0b72fdeec9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Koz=C5=82owski?= Date: Wed, 2 Sep 2026 20:06:52 +0200 Subject: [PATCH] Document how to build a lossless transport MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The @lossless section said a lossless transport is required, named the runtime as one, and pointed at parseLossless / stringifyLossless — but in one sentence, with no code and no list of where the substitution has to happen. Since nothing in the generated code can detect a lossy transport, that is the one place where being terse costs a silent bug: the schema validates, the field is a number, and it is the wrong number. So: a "Building a lossless transport" subsection, and a first line that lets anyone using the runtime's fetchTransport stop reading. It enumerates all four JSON boundaries rather than the two obvious ones — request body, response body, *and* per-line ndjson in both directions. All four exist in runtime/src (fetch.ts:90,150,241 and ndjson.ts:21,53), so a transport that fixes only the unary pair still rounds a @lossless member inside a stream. Two specific traps, both from reading the reference implementation: - res.json() cannot be fixed from outside — the rounding happens inside it, before the caller holds the value. Read text() and parse it. - StreamTransportResponse.stream is specified as already-parsed elements, so parsing them losslessly is the transport's job; no later schema can undo it. The doc comment on that field said "already JSON.parse'd", which named the exact function a transport author must not use there. Fixed, and the lossless requirement stated where an implementor will actually read it. Also documents the two body-reading cases that are not about precision but that the generated client depends on — an empty body (204, or no output members) is not valid JSON, and a non-2xx body may not be JSON at all — with the readBody from fetch.ts, since throwing on either turns a useful status into a parse error. Docs and one doc comment; no behaviour change. Runtime typecheck + 46 tests pass. --- README.md | 69 ++++++++++++++++++++++++++++++++++++++++---- runtime/src/types.ts | 10 +++++-- 2 files changed, 71 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 0ed6ba0..cc2750b 100644 --- a/README.md +++ b/README.md @@ -300,11 +300,70 @@ would lose precision surfaces as its exact decimal string. On the way out the ge converts the member to a `bigint`, which the serializer writes as a bare numeric literal — so `number`, `string` and `bigint` are all accepted, and all reach the wire unquoted. -**This requires a lossless transport.** `@polyvariant/smithy-ts-runtime` is one. A transport built -on plain `JSON.parse` / `JSON.stringify` cannot honor the trait: `JSON.parse` will have rounded the -value before the schema runs, and `JSON.stringify` throws on a `bigint`. A hand-rolled transport -should use the exported `parseLossless` / `stringifyLossless` in place of the built-ins. Nothing in -the generated code can detect the difference, so this is on you to wire up. +**This requires a lossless transport.** `@polyvariant/smithy-ts-runtime` provides one: if you use +its `fetchTransport`, `@lossless` already works and there is nothing to wire up. The next section is +only for a transport you write yourself. + +### Building a lossless transport + +A transport built on plain `JSON.parse` / `JSON.stringify` cannot honor the trait: `JSON.parse` will +have rounded the value before the schema runs, and `JSON.stringify` throws outright on the `bigint` +the generated client hands it. Nothing in the generated code can detect which you used, so getting +this wrong fails *silently* on the read path — the schema validates, the field is a `number`, and it +is the wrong number. + +The runtime exports the two replacements, so this is a substitution rather than a rewrite: + +```ts +import { parseLossless, stringifyLossless } from '@polyvariant/smithy-ts-runtime' +``` + +They are drop-in. `parseLossless` returns a `number` for every value that round-trips exactly, so +ordinary fields are untouched and only a value that would lose precision comes back as its exact +decimal string — which is why `@lossless` members admit both. `stringifyLossless` writes a `bigint` +as a bare numeric literal. + +Substitute at **every** JSON boundary the transport has. There are more than the obvious two: + +| Boundary | Use | +| --- | --- | +| Reading a response body | `parseLossless(await res.text())` | +| Writing a request body | `stringifyLossless(req.body)` | +| Reading an ndjson response line | `parseLossless(line)`, per line | +| Writing an ndjson request element | `stringifyLossless(element) + '\n'`, per element | + +**Do not use `res.json()`.** It is the natural way to read a body and it cannot be fixed from the +outside: the rounding happens inside it, before you ever hold the value. Read `text()` and parse it +yourself. + +The last two rows apply only if the transport implements `StreamTransport`. Note that +`StreamTransportResponse.stream` is specified as *already-parsed* ndjson elements — so parsing them +losslessly is the transport's job, not something the generated client can do afterwards. + +Reading a body has two cases that are not about precision but are easy to get wrong, and the +generated client depends on both: + +```ts +const readBody = async (res: Response): Promise => { + const text = await res.text() + if (text.length === 0) return undefined // 204, or an operation with no output members + try { + return parseLossless(text) + } catch { + return text // a proxy's HTML error page: let the status reach the caller + } +} +``` + +An empty body is not valid JSON, and a non-2xx body is not necessarily JSON at all; throwing on +either turns a useful status code into a parse error. + +Nothing else needs special handling — in particular, a `@lossless` member bound to a path, query or +header parameter is passed through as-is by the generated client, since those are strings on the wire +and were never lossy. + +For a complete reference, `runtime/src/fetch.ts` and `runtime/src/ndjson.ts` in this repository are +these substitutions over an otherwise ordinary `fetch` transport. The trait is member-scoped, not shape-scoped: whether a field can exceed the safe range is a property of that field, and the same numeric shape is usually reused for values that stay well diff --git a/runtime/src/types.ts b/runtime/src/types.ts index 2017a4d..7bf4dfb 100644 --- a/runtime/src/types.ts +++ b/runtime/src/types.ts @@ -69,9 +69,13 @@ export interface StreamTransportResponse { status: number headers: Record /** The deframed response body — present exactly when the operation streams - * its output and the status was 2xx. Already `JSON.parse`d for `'ndjson'`, - * raw `Uint8Array` chunks for `'binary'`. The generated client validates - * ndjson elements against the operation's schema. */ + * its output and the status was 2xx. Already parsed for `'ndjson'`, raw + * `Uint8Array` chunks for `'binary'`. The generated client validates ndjson + * elements against the operation's schema. + * + * Parse ndjson lines with `parseLossless`, not `JSON.parse`: elements are + * parsed here, before the client sees them, so a `@lossless` member would + * already have been rounded by the time any schema runs. */ stream?: AsyncIterable /** The fully-read body, for a non-2xx response (so declared errors can be * parsed and thrown) or an operation with a unary response body. */