Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/fta-v3.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,18 @@ on:
- 'main'
paths:
- 'packages/stack/src/eql/v3/**'
# Shared match-index defaults live outside src/eql/v3 but shape every
# emitted v3 match block (load-bearing `k`/`m` ciphertext params), so edits
# here must trigger the v3 gate too.
- 'packages/stack/src/schema/match-defaults.ts'
- 'packages/stack/package.json'
- '.github/workflows/fta-v3.yml'
pull_request:
branches:
- "**"
paths:
- 'packages/stack/src/eql/v3/**'
- 'packages/stack/src/schema/match-defaults.ts'
Comment on lines 21 to +26

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Trigger the gate when the workflow itself changes.

push covers .github/workflows/fta-v3.yml, but pull_request does not. That means PRs can change this CI gate without ever running the updated workflow before merge.

Suggested fix
   pull_request:
     branches:
       - "**"
     paths:
       - 'packages/stack/src/eql/v3/**'
       - 'packages/stack/src/schema/match-defaults.ts'
+      - '.github/workflows/fta-v3.yml'
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
pull_request:
branches:
- "**"
paths:
- 'packages/stack/src/eql/v3/**'
- 'packages/stack/src/schema/match-defaults.ts'
pull_request:
branches:
- "**"
paths:
- 'packages/stack/src/eql/v3/**'
- 'packages/stack/src/schema/match-defaults.ts'
- '.github/workflows/fta-v3.yml'
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/fta-v3.yml around lines 21 - 26, Add the workflow file
itself to the pull_request path filters in fta-v3 so changes to
.github/workflows/fta-v3.yml also trigger the gate; update the pull_request
paths alongside the existing packages/stack/src/eql/v3 and schema/match-defaults
entries so the workflow validates its own changes before merge.

- 'packages/stack/package.json'
- '.github/workflows/fta-v3.yml'

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -147,8 +147,10 @@ Implementation therefore includes a **baseline step**:
locally.
2. Record the reported mutation score for `eql/v3`.
3. Set `thresholds.break` just **below** the measured score (a small buffer, the
way FTA sets `--score-cap 72` against a current 71.08). This ensures the
current state passes while any regression that lowers the score fails the gate.
way FTA sets `--score-cap 69` against a current worst-file score of 68.00 —
re-baselined from the pre-split monolith's 71.08/72 after `eql/v3` was split
into per-file modules). This ensures the current state passes while any
regression that lowers the score fails the gate.
4. Set `high`/`low` to reasonable display bands (do not affect pass/fail).

If the measured baseline is very low (tests are weak), surface that to the user
Expand Down
76 changes: 49 additions & 27 deletions packages/stack/__tests__/v3-matrix/matrix-live-pg.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,14 @@
* and not the other. Dispatch mirrors the priority `resolveIndexType` itself
* uses (match > unique > ore > none):
* - match (text_match, text_search): `eql_v3.match_term` + `bloom_filter`
* - eq (*_eq domains): `eql_v3.eq_term` + `hmac_256`
* - ord (*_ord / *_ord_ore domains): `eql_v3.ord_term` + `ore_block_256`,
* queried with `queryType:'equality'` — the exact path Part A fixed. Most
* ord-tier domains (all but text) have no `eq_term` at all in the real
* `eql_v3` SQL (verified against the fixture), so this is not a stylistic
* choice: it is the only equality path that exists for them.
* - eq (any `unique`/`hm` domain): `eql_v3.eq_term` + `hmac_256`
* - ord (any `ore`/`ob` domain): `eql_v3.ord_term` + `ore_block_256`.
* Pure-ORE domains (numeric/date `*_ord`/`*_ord_ore`) are queried with
* `queryType:'equality'` — the equality-via-ORE path Part A fixed — because
* `ob` is their only index and they have no `eq_term` at all in the real
* `eql_v3` SQL (verified against the fixture). Text order domains carry BOTH
* `hm` and `ob`, so they run the eq proof AND the ord proof; their `ob` term
* is built with `queryType:'orderAndRange'` (equality would resolve to `hm`).
* - storage (no index): no query is possible; proves the ciphertext, cast to
* THIS SPECIFIC Postgres domain type, survives a real INSERT/SELECT and
* still decrypts — the one thing the FFI-only round-trip can't show.
Expand Down Expand Up @@ -85,34 +87,45 @@ const columns = Object.fromEntries(
const table = encryptedTable(TABLE_NAME, columns as never)

/**
* The one proof each domain's configured indexes call for — mirrors the
* priority `resolveIndexType`/`inferIndexType` themselves use: match wins over
* unique wins over ore. `text_search` carries all three but gets the match
* proof (its distinguishing, richest capability); the plain `*_eq` domains get
* the eq proof; every `*_ord`/`*_ord_ore` domain (including the text ones,
* which also have an `eq_term` but are queried the same way as their
* non-text siblings for consistency) gets the equality-via-ORE proof.
* Which proofs a domain's configured indexes call for. Unlike a single-kind
* classifier these lists are NOT mutually exclusive — a domain runs EVERY proof
* its indexes support:
*
* - eq (`hm`): every domain carrying a `unique` index → `eq_term`/`hmac_256`.
* - ord (`ob`): every domain carrying an `ore` index → `ord_term`/`ore_block_256`.
* - match (`bf`): every domain carrying a `match` index → `match_term`/`bloom_filter`.
* - storage: a domain with NO index — only the ciphertext round-trip proof.
*
* Text order domains (`text_ord`/`text_ord_ore`) carry BOTH `unique` and `ore`,
* so they appear in `eqDomains` AND `ordDomains` and run both proofs — a
* wrong-valued `ob` would otherwise slip through an eq-only check (text equality
* is HMAC-based, so `queryType:'equality'` on them resolves to `hm`, never `ob`;
* their ord term is built with `queryType:'orderAndRange'` below). `text_search`
* also carries all three indexes but is deliberately exercised by the match
* proof ALONE — its distinguishing, richest capability and the one canonical
* example per tier — so it is excluded from the eq/ord lists via `!match`.
*/
type ProofKind = 'match' | 'eq' | 'ord' | 'storage'
function proofKindFor(indexes: DomainSpec['indexes']): ProofKind {
const idx = indexes ?? {}
if (idx.match) return 'match'
if (idx.unique) return 'eq'
if (idx.ore) return 'ord'
return 'storage'
}
const hasIndex = (
indexes: DomainSpec['indexes'],
key: 'unique' | 'ore' | 'match',
): boolean => Boolean((indexes ?? {})[key])

const matchDomains = domains.filter(
([, spec]) => proofKindFor(spec.indexes) === 'match',
const matchDomains = domains.filter(([, spec]) =>
hasIndex(spec.indexes, 'match'),
)
const eqDomains = domains.filter(
([, spec]) => proofKindFor(spec.indexes) === 'eq',
([, spec]) =>
hasIndex(spec.indexes, 'unique') && !hasIndex(spec.indexes, 'match'),
)
const ordDomains = domains.filter(
([, spec]) => proofKindFor(spec.indexes) === 'ord',
([, spec]) =>
hasIndex(spec.indexes, 'ore') && !hasIndex(spec.indexes, 'match'),
)
const storageDomains = domains.filter(
([, spec]) => proofKindFor(spec.indexes) === 'storage',
([, spec]) =>
!hasIndex(spec.indexes, 'unique') &&
!hasIndex(spec.indexes, 'ore') &&
!hasIndex(spec.indexes, 'match'),
)
const textOreDomains = domains.filter(
([t]) =>
Expand Down Expand Up @@ -204,13 +217,22 @@ beforeAll(async () => {
)
}
for (const [t, spec] of ordDomains) {
// Pure-ORE domains (numeric/date) answer equality via ORE, so
// `queryType:'equality'` resolves to the `ob` term — the exact
// equality-via-ORE path Part A fixed. Text order domains ALSO carry `hm`,
// where `equality` resolves to HMAC by the shared `unique > … > ore`
// priority; force `orderAndRange` there so the term still carries the `ob`
// this proof extracts with `ore_block_256`.
const queryType = hasIndex(spec.indexes, 'unique')
? 'orderAndRange'
: 'equality'
ordTerms[slug(t)] = unwrapResult(
await client.encryptQuery(
spec.samples[0] as never,
{
table,
column: columnRef(t),
queryType: 'equality',
queryType,
} as never,
),
)
Expand Down
2 changes: 1 addition & 1 deletion packages/stack/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@
"prebuild": "node -e \"require('node:fs').rmSync('dist',{recursive:true,force:true})\"",
"build": "tsup",
"dev": "tsup --watch",
"analyze:complexity": "fta src/eql/v3 --score-cap 72",
"analyze:complexity": "fta src/eql/v3 --score-cap 69",
"db:eql-v3:install": "tsx scripts/install-eql-v3.ts",
"test": "vitest run",
"test:types": "vitest --run --typecheck.only",
Expand Down
37 changes: 33 additions & 4 deletions packages/stack/src/encryption/v3.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import type {
V3EncryptedModel,
V3ModelInput,
} from '@/eql/v3'
import type { EncryptionError } from '@/errors'
import { type EncryptionError, EncryptionErrorTypes } from '@/errors'
import type { LockContextInput } from '@/identity'
import type {
BulkDecryptPayload,
Expand Down Expand Up @@ -164,8 +164,34 @@ function rowReconstructor(
*/
export function typedClient<const S extends readonly AnyV3Table[]>(
client: EncryptionClient,
..._schemas: S
...schemas: S
): TypedEncryptionClient<S> {
// Precompute one row reconstructor per schema table at construction. This runs
// each table's `build()` — which throws on duplicate DB column names — ONCE,
// here, off the Result-returning decrypt path. `decryptModel`/
// `bulkDecryptModels` therefore never call `build()` (whose throw would surface
// as a promise rejection and break their `Promise<Result<…>>` contract) and no
// longer rebuild the row-invariant config on every call.
const reconstructors = new Map<
AnyV3Table,
(row: Record<string, unknown>) => Record<string, unknown>
>()
for (const table of schemas) {
reconstructors.set(table, rowReconstructor(table))
}

// A table not among the schemas this client was initialized with (only
// reachable by bypassing the `Table extends S[number]` type constraint) has no
// precomputed reconstructor. Return a Result failure rather than building one
// inline, which could throw and reject the Result-shaped decrypt promise.
const unknownTableFailure: { failure: EncryptionError } = {
failure: {
type: EncryptionErrorTypes.DecryptionError,
message:
'[eql/v3]: decryptModel received a table this client was not initialized with — pass the same table object(s) given to EncryptionV3/typedClient',
},
}

return {
encrypt: (plaintext, opts) =>
client.encrypt(plaintext as never, opts as never),
Expand All @@ -177,16 +203,19 @@ export function typedClient<const S extends readonly AnyV3Table[]>(
client.bulkEncryptModels(input as never, table as never) as never,
decrypt: (encrypted) => client.decrypt(encrypted),
decryptModel: async (input, table, lockContext) => {
const reconstruct = reconstructors.get(table)
if (!reconstruct) return unknownTableFailure as never
const op = client.decryptModel(input as never)
const result = await (lockContext ? op.withLockContext(lockContext) : op)
if (result.failure) return result as never
return { data: rowReconstructor(table)(result.data) } as never
return { data: reconstruct(result.data) } as never
},
bulkDecryptModels: async (input, table, lockContext) => {
const reconstruct = reconstructors.get(table)
if (!reconstruct) return unknownTableFailure as never
const op = client.bulkDecryptModels(input as never)
const result = await (lockContext ? op.withLockContext(lockContext) : op)
if (result.failure) return result as never
const reconstruct = rowReconstructor(table)
return {
data: result.data.map((row) =>
reconstruct(row as Record<string, unknown>),
Expand Down
37 changes: 17 additions & 20 deletions packages/stack/src/eql/v3/columns.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import {
type BuiltMatchIndexOpts,
cloneMatchOpts,
defaultMatchOpts,
resolveMatchOpts,
} from '@/schema/match-defaults'

/**
Expand Down Expand Up @@ -422,33 +423,29 @@ export class EncryptedTextSearchColumn extends EncryptedV3Column<
* on for this type. Merge semantics mirror v2's `opts?.x ?? default`.
*/
freeTextSearch(opts?: MatchIndexOpts): this {
// A fresh defaults object per call supplies the `?? ` fallbacks, so no
// nested default object is ever shared into `this.matchOpts` by reference.
const defaults = defaultMatchOpts()
// Clone-on-write: deep-copy the nested tokenizer / token_filters when
// storing them so a caller mutating their own opts object between
// freeTextSearch(opts) and build() cannot leak into the emitted config.
this.matchOpts = cloneMatchOpts({
tokenizer: opts?.tokenizer ?? defaults.tokenizer,
token_filters: opts?.token_filters ?? defaults.token_filters,
k: opts?.k ?? defaults.k,
m: opts?.m ?? defaults.m,
include_original: opts?.include_original ?? defaults.include_original,
})
// Shared merge+clone (schema/match-defaults) — one source of truth with the
// v2 `freeTextSearch()` builder. `resolveMatchOpts` merges each key over the
// per-call defaults and deep-clones, so a caller mutating their own opts
// object between `freeTextSearch(opts)` and `build()` cannot leak into the
// emitted config (clone-on-write).
this.matchOpts = resolveMatchOpts(opts)
return this
}

/** Emit the encrypt-config column. Byte-identical to a v2 equality+order+match column. */
override build(): ColumnSchema {
// Deep-clone the match block so the returned config NEVER aliases this
// builder's internal `matchOpts` (or any caller-supplied opts merged into
// it). A caller mutating the returned object cannot corrupt this builder's
// state or another column's defaults.
// Derive `cast_as` + the `unique`/`ore` blocks from the shared
// capability→index mapping (this domain's TEXT_SEARCH capabilities produce
// exactly `{ unique, ore, match }`), then override ONLY `match` with this
// builder's tuned opts. Hand-writing the unique/ore shape here would let it
// silently drift from `indexesForCapabilities` — the exact divergence class
// behind the text_ord `hm`-index bug. Deep-clone the match block so the
// returned config never aliases this builder's internal `matchOpts`.
const base = super.build()
return {
cast_as: 'string',
...base,
indexes: {
unique: { token_filters: [] },
ore: {},
...base.indexes,
match: cloneMatchOpts(this.matchOpts),
},
}
Expand Down
19 changes: 11 additions & 8 deletions packages/stack/src/eql/v3/table.ts
Original file line number Diff line number Diff line change
Expand Up @@ -211,11 +211,14 @@ export type V3EncryptedModel<Table extends AnyV3Table, T> = {
: T[K]
}

/** The decrypted result model: schema columns become their plaintext type, others pass through. */
export type V3DecryptedModel<Table extends AnyV3Table, T> = {
[K in keyof T]: K extends keyof InferPlaintext<Table>
? null extends T[K]
? InferPlaintext<Table>[K] | null
: InferPlaintext<Table>[K]
: T[K]
}
/**
* The decrypted result model: schema columns become their plaintext type, others
* pass through. Structurally identical to {@link V3ModelInput} — decrypt yields
* the same plaintext shape encrypt accepts — so it is aliased rather than
* re-declared to keep the input and output shapes from silently drifting when
* one copy is edited. The distinct name is kept for call-site readability.
*/
export type V3DecryptedModel<Table extends AnyV3Table, T> = V3ModelInput<
Table,
T
>
17 changes: 6 additions & 11 deletions packages/stack/src/schema/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { z } from 'zod'
import type { BuildableTable, Encrypted } from '@/types'
import { defaultMatchOpts } from './match-defaults'
import { resolveMatchOpts } from './match-defaults'

// ------------------------
// Zod schemas
Expand Down Expand Up @@ -352,16 +352,11 @@ export class EncryptedColumn {
* ```
*/
freeTextSearch(opts?: MatchIndexOpts) {
// Shared defaults (schema/match-defaults) — one source of truth with the
// EQL v3 domain builders. The factory returns fresh nested objects.
const defaults = defaultMatchOpts()
this.indexesValue.match = {
tokenizer: opts?.tokenizer ?? defaults.tokenizer,
token_filters: opts?.token_filters ?? defaults.token_filters,
k: opts?.k ?? defaults.k,
m: opts?.m ?? defaults.m,
include_original: opts?.include_original ?? defaults.include_original,
}
// Shared merge+clone (schema/match-defaults) — one source of truth with the
// EQL v3 domain builders. `resolveMatchOpts` deep-clones, so a caller
// mutating their own `opts` (or its nested tokenizer/token_filters) after
// this call cannot leak into the stored schema.
this.indexesValue.match = resolveMatchOpts(opts)
return this
}

Expand Down
22 changes: 22 additions & 0 deletions packages/stack/src/schema/match-defaults.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,25 @@ export function cloneMatchOpts(opts: BuiltMatchIndexOpts): BuiltMatchIndexOpts {
token_filters: opts.token_filters.map((f) => ({ ...f })),
}
}

/**
* Resolve user-supplied `freeTextSearch(opts)` input into a fully-built match
* block: each provided key replaces its default, omitted keys keep the default
* (`opts?.x ?? default.x`). The single source of truth for that five-field merge
* shared by the v2 `freeTextSearch()` builder and the v3 domain builders.
*
* The result is deep-cloned ({@link cloneMatchOpts}) so a caller mutating their
* own `opts` object (or its nested `tokenizer`/`token_filters`) after this call
* can never leak into the stored builder state or the emitted config — clone-on-
* write for both builders, not just v3.
*/
export function resolveMatchOpts(opts?: MatchIndexOpts): BuiltMatchIndexOpts {
const defaults = defaultMatchOpts()
return cloneMatchOpts({
tokenizer: opts?.tokenizer ?? defaults.tokenizer,
token_filters: opts?.token_filters ?? defaults.token_filters,
k: opts?.k ?? defaults.k,
m: opts?.m ?? defaults.m,
include_original: opts?.include_original ?? defaults.include_original,
})
}
Loading