diff --git a/.changeset/eql-v3-drizzle.md b/.changeset/eql-v3-drizzle.md new file mode 100644 index 000000000..aa45af42a --- /dev/null +++ b/.changeset/eql-v3-drizzle.md @@ -0,0 +1,17 @@ +--- +"@cipherstash/stack": minor +--- + +Add EQL v3 Drizzle support at `@cipherstash/stack/eql/v3/drizzle`. A Drizzle-native +`types` namespace (same PascalCase names as `@cipherstash/stack/eql/v3`) declares +encrypted columns whose Postgres type is the semantic `public.`; the concrete +type drives the legal query operators. `createEncryptionOperatorsV3` provides +capability-checked `eq`/`ne`/`gt`/`gte`/`lt`/`lte`/`between`/`contains`/`inArray`/ +`asc`/`desc`/`and`/`or` that emit the latest two-argument `eql_v3` SQL functions with +full-envelope operands, and +`extractEncryptionSchemaV3` rebuilds the schema for `EncryptionV3`. The existing v2 +`@cipherstash/stack/drizzle` integration is unchanged. + +The v3 text-search helper is `contains`; obsolete `like`/`ilike` helpers are not +exposed because v3 free-text search is token containment rather than SQL wildcard +matching. diff --git a/packages/stack/README.md b/packages/stack/README.md index 1a57a8cb6..7d27e88fd 100644 --- a/packages/stack/README.md +++ b/packages/stack/README.md @@ -366,6 +366,87 @@ For columns with `searchableJson: true`, three JSONB operators are available: These operators encrypt the JSON path selector using the `steVecSelector` query type and cast it to `eql_v2_encrypted` for use with the EQL PostgreSQL functions. +### EQL v3 Concrete-Type Columns (Drizzle) + +The `@cipherstash/stack/eql/v3/drizzle` module targets EQL v3, where each encrypted column is a **concrete Postgres domain type** (`eql_v3.text_eq`, `eql_v3.integer_ord`, …). Instead of toggling capability flags, you pick a concrete type from the `types` namespace and its query capabilities are fixed by that choice. The v2 `@cipherstash/stack/drizzle` module above is unchanged — this is an additive export. + +Declare a Drizzle table using the `types` factories. The suffix encodes the capability: `*Eq` (equality), `*Ord` (order + range, which also covers equality), `*Match` / `TextSearch` (free-text search), and the bare name (e.g. `Text`, `Bigint`) is storage-only. + +```ts +import { pgTable, integer } from "drizzle-orm/pg-core" +import { drizzle } from "drizzle-orm/postgres-js" +import { + types, + createEncryptionOperatorsV3, + extractEncryptionSchemaV3, +} from "@cipherstash/stack/eql/v3/drizzle" +import { EncryptionV3 } from "@cipherstash/stack/v3" + +// Capabilities come from the concrete type — no flags to configure. +const users = pgTable("users", { + id: integer("id").primaryKey().generatedAlwaysAsIdentity(), + email: types.TextEq("email"), // equality: eq / ne / inArray + age: types.IntegerOrd("age"), // order + range: gt/gte/lt/lte, between, asc/desc + bio: types.TextMatch("bio"), // free-text search: contains + balance: types.Bigint("balance"), // storage only (no query capability) +}) +``` + +Derive the v3 schema from the table, build the typed client, and create the capability-checked operators: + +```ts +const usersSchema = extractEncryptionSchemaV3(users) +const client = await EncryptionV3({ schemas: [usersSchema] }) +const ops = createEncryptionOperatorsV3(client) + +const db = drizzle({ client: sqlClient }) +``` + +The operators auto-encrypt their operands and validate them against the column's concrete type. Applying an operator the type doesn't support throws `EncryptionOperatorError`: + +```ts +// Equality — email is TextEq +const exact = await db.select().from(users) + .where(await ops.eq(users.email, "alice@example.com")) + +// Range + ordering — age is IntegerOrd +const adults = await db.select().from(users) + .where(await ops.gte(users.age, 18)) + .orderBy(ops.asc(users.age)) + +const midBand = await db.select().from(users) + .where(await ops.between(users.age, 25, 40)) + +// Set membership — built on equality +const listed = await db.select().from(users) + .where(await ops.inArray(users.email, ["alice@example.com", "bob@example.com"])) + +// Free-text token containment — bio is TextMatch +const coffee = await db.select().from(users) + .where(await ops.contains(users.bio, "coffee")) +``` + +Rows are **pre-encrypted** with `client.bulkEncryptModels(...)` before they reach `db.insert(...).values(...)` — Drizzle never sees plaintext. `Bigint` columns take a native JS `bigint`: + +```ts +const rows = await client.bulkEncryptModels( + [ + { email: "alice@example.com", age: 30, bio: "climbing and coffee", balance: 100_000n }, + { email: "bob@example.com", age: 41, bio: "cycling and coffee", balance: 250_000n }, + ], + usersSchema, +) +if (rows.failure) throw new Error(rows.failure.message) + +await db.insert(users).values(rows.data) +``` + +Notes: + +- **Free-text search uses `ops.contains`** (token containment on a `match` index), not SQL `like` / `ilike`. It matches whole indexed tokens, so `contains(users.bio, "coffee")` finds rows whose `bio` contains the token `coffee`. +- **The concrete type defines the legal operators.** `TextEq` supports `eq` / `ne` / `inArray` / `notInArray`; `*Ord` types add `gt` / `gte` / `lt` / `lte` / `between` / `notBetween` and `asc` / `desc`; `*Match` and `TextSearch` add `contains`; a bare `Text` / `Integer` / `Bigint` column is storage-only. Using an unsupported operator throws `EncryptionOperatorError`. +- Combine conditions with `ops.and` / `ops.or`, and do NULL checks with `ops.isNull` / `ops.isNotNull` (the where-clause operators are `async` and must be `await`ed; `ops.asc` / `ops.desc` are synchronous). + ## Identity-Aware Encryption Lock encryption to a specific user by requiring a valid JWT for decryption. @@ -664,6 +745,8 @@ csValue(valueName) // returns ProtectValue (for nested values) | `@cipherstash/stack/secrets` | `Secrets` class and secrets types | | `@cipherstash/stack/client` | Client-safe exports (schema builders and types only - no native FFI) | | `@cipherstash/stack/types` | All TypeScript types (`Encrypted`, `Decrypted`, `ClientConfig`, `EncryptionClientConfig`, query types, etc.) | +| `@cipherstash/stack/v3` | `EncryptionV3` typed client plus the EQL v3 authoring DSL (`encryptedTable`, `types`, v3 type helpers) | +| `@cipherstash/stack/eql/v3/drizzle` | EQL v3 Drizzle integration: `types` column factories, `createEncryptionOperatorsV3`, `extractEncryptionSchemaV3`, `makeEqlV3Column`, `EncryptionOperatorError` | ## Migration from @cipherstash/protect diff --git a/packages/stack/__tests__/bulk-encrypt-validation.test.ts b/packages/stack/__tests__/bulk-encrypt-validation.test.ts new file mode 100644 index 000000000..a267e0700 --- /dev/null +++ b/packages/stack/__tests__/bulk-encrypt-validation.test.ts @@ -0,0 +1,107 @@ +import { describe, expect, it } from 'vitest' +import { + BulkEncryptOperation, + BulkEncryptOperationWithLockContext, +} from '@/encryption/operations/bulk-encrypt' +import type { + BuildableColumn, + BuildableTable, + BulkEncryptPayload, + Client, +} from '@/types' + +/** + * `EncryptOperation` rejects NaN / ±Infinity / out-of-int64 `bigint` values + * client-side, before they reach protect-ffi (whose behaviour on such a value + * is unobservable — see `helpers/validation.ts`). `BulkEncryptOperation` must + * enforce the same contract: any caller that batches instead of looping — the + * v3 Drizzle `inArray`, for one — otherwise silently loses the guard. + * + * The stub client is never reached: validation must throw first. If a case + * here ever calls into protect-ffi, the fake client surfaces it as a different + * error than the one asserted. + */ +const client = {} as Client + +const column: BuildableColumn = { + getName: () => 'age', + build: () => ({}) as never, +} + +const table: BuildableTable = { + tableName: 'users', + build: () => ({ tableName: 'users', columns: {} }), +} + +const payload = (...values: unknown[]): BulkEncryptPayload => + values.map((plaintext) => ({ plaintext })) as BulkEncryptPayload + +const lockContext = { + ctsToken: { accessToken: 'token' }, +} as never + +describe('BulkEncryptOperation numeric validation', () => { + it.each([ + [Number.NaN, 'Cannot encrypt NaN value'], + [Number.POSITIVE_INFINITY, 'Cannot encrypt Infinity value'], + [Number.NEGATIVE_INFINITY, 'Cannot encrypt Infinity value'], + [2n ** 70n, 'Cannot encrypt bigint value out of int64 range'], + [-(2n ** 70n), 'Cannot encrypt bigint value out of int64 range'], + ])('rejects %s before reaching the FFI', async (value, message) => { + const op = new BulkEncryptOperation(client, payload(value), { + column, + table, + }) + + const result = await op.execute() + + expect(result.failure?.message).toContain(message) + }) + + it('rejects an invalid value anywhere in the list, not just the first', async () => { + const op = new BulkEncryptOperation(client, payload(30, 42, Number.NaN), { + column, + table, + }) + + const result = await op.execute() + + expect(result.failure?.message).toContain('Cannot encrypt NaN value') + }) + + it('still passes null entries through without tripping validation', async () => { + const op = new BulkEncryptOperation(client, [{ plaintext: null }], { + column, + table, + }) + + const result = await op.execute() + + expect(result.failure).toBeUndefined() + expect(result.data).toEqual([{ id: undefined, data: null }]) + }) + + it('accepts in-range bigints at the int64 boundary', async () => { + const op = new BulkEncryptOperation(client, payload(9223372036854775807n), { + column, + table, + }) + + const result = await op.execute() + + // Validation passes, so the stub client is reached — proving the boundary + // value was NOT rejected by the numeric guard. + expect(result.failure?.message).not.toContain('out of int64 range') + }) + + it('enforces the same guard on the lock-context variant', async () => { + const op = new BulkEncryptOperationWithLockContext( + new BulkEncryptOperation(client, payload(Number.NaN), { column, table }), + lockContext, + ) + + const result = await op.execute() + + expect(result.failure?.message).toContain('Cannot encrypt NaN value') + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/bigint.test.ts b/packages/stack/__tests__/drizzle-v3/bigint.test.ts new file mode 100644 index 000000000..e33994591 --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/bigint.test.ts @@ -0,0 +1,99 @@ +import { type SQL, sql } from 'drizzle-orm' +import { integer, PgDialect, pgTable } from 'drizzle-orm/pg-core' +import { describe, expect, it, vi } from 'vitest' +import { getEqlV3Column, isEqlV3Column } from '@/eql/v3/drizzle/column' +import { + createEncryptionOperatorsV3, + EncryptionOperatorError, +} from '@/eql/v3/drizzle/operators' +import { types } from '@/eql/v3/drizzle/types' + +// A representative encrypted envelope — what `client.encrypt` actually returns +// and what a bigint column stores. Deliberately NOT a plaintext bigint. +const TERM = { c: 'ct', v: 1 } +const TERM_JSON = JSON.stringify(TERM) + +function chainable(result: unknown) { + const op = result as { + withLockContext: ReturnType + audit: ReturnType + } + op.withLockContext = vi.fn(() => op) + op.audit = vi.fn(() => op) + return op +} + +function setup() { + const encrypt = vi.fn(() => chainable(Promise.resolve({ data: TERM }))) + const ops = createEncryptionOperatorsV3({ encrypt }) + const dialect = new PgDialect() + const render = (s: SQL) => dialect.sqlToQuery(s) + return { ops, encrypt, render } +} + +// A statically-typed table via the drizzle `types` namespace (no dynamic +// matrix, no casts) — the column set is known at compile time. +const accounts = pgTable('accounts', { + id: integer().primaryKey(), + balance: types.BigintOrd('balance'), + ledgerId: types.BigintEq('ledger_id'), + archived: types.Bigint('archived'), +}) + +describe('v3 drizzle bigint columns', () => { + it('detects a bigint column and reports its concrete public.bigint domain', () => { + const storage = types.Bigint('n') + const ord = types.BigintOrd('n_ord') + + expect(isEqlV3Column(storage)).toBe(true) + expect(getEqlV3Column('n', storage)?.getEqlType()).toBe('public.bigint') + expect(getEqlV3Column('n_ord', ord)?.getEqlType()).toBe('public.bigint_ord') + }) + + it('emits the concrete public.bigint* SQL type through pgTable', () => { + expect(accounts.balance.getSQLType()).toBe('public.bigint_ord') + expect(accounts.ledgerId.getSQLType()).toBe('public.bigint_eq') + expect(accounts.archived.getSQLType()).toBe('public.bigint') + }) + + it('encrypts a native bigint operand for eq without JSON-stringifying it', async () => { + const { ops, encrypt, render } = setup() + // A value beyond Number.MAX_SAFE_INTEGER to prove the bigint is passed + // through untouched (a JSON.stringify of a bigint would have thrown). + const value = 9223372036854775807n + const q = render(await ops.eq(accounts.ledgerId, value)) + + expect(q.sql).toContain('eql_v3.eq("accounts"."ledger_id", $1::jsonb)') + expect(q.params).toEqual([TERM_JSON]) + expect(encrypt.mock.calls[0]?.[0]).toBe(value) + }) + + it('emits ordering and range operators for a bigint_ord column', async () => { + const { ops, render } = setup() + + const gt = render(await ops.gt(accounts.balance, 10n)) + expect(gt.sql).toContain('eql_v3.gt("accounts"."balance", $1::jsonb)') + + const between = render(await ops.between(accounts.balance, -5n, 5n)) + expect(between.sql).toContain('eql_v3.gte("accounts"."balance", $1::jsonb)') + expect(between.sql).toContain('eql_v3.lte("accounts"."balance", $2::jsonb)') + + const asc = render(ops.asc(accounts.balance)) + expect(asc.sql).toContain('eql_v3.ord_term("accounts"."balance")') + }) + + it('rejects equality/ordering on a storage-only bigint column', async () => { + const { ops } = setup() + await expect(ops.eq(accounts.archived, 1n)).rejects.toBeInstanceOf( + EncryptionOperatorError, + ) + expect(() => ops.asc(accounts.archived)).toThrow(EncryptionOperatorError) + }) + + it('rejects a bare SQL expression that is not an encrypted column', async () => { + const { ops } = setup() + await expect(ops.eq(sql`1`, 1n)).rejects.toBeInstanceOf( + EncryptionOperatorError, + ) + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/codec.test.ts b/packages/stack/__tests__/drizzle-v3/codec.test.ts new file mode 100644 index 000000000..698b34821 --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/codec.test.ts @@ -0,0 +1,31 @@ +import { describe, expect, it } from 'vitest' +import { v3FromDriver, v3ToDriver } from '@/eql/v3/drizzle/codec' + +describe('v3 codec', () => { + it('serialises an object to a jsonb string', () => { + expect(v3ToDriver({ v: 1, c: 'ct' })).toBe('{"v":1,"c":"ct"}') + }) + + it('maps null/undefined to SQL NULL (JS null), never the JSON null literal', () => { + expect(v3ToDriver(null)).toBeNull() + expect(v3ToDriver(undefined)).toBeNull() + }) + + it('parses a jsonb string back to an object', () => { + expect(v3FromDriver('{"v":1,"c":"ct"}')).toEqual({ v: 1, c: 'ct' }) + }) + + it('passes an already-parsed object through unchanged', () => { + const obj = { v: 1 } + expect(v3FromDriver(obj)).toBe(obj) + }) + + it('normalises null/undefined to SQL NULL (JS null) on read', () => { + expect(v3FromDriver(null)).toBeNull() + expect(v3FromDriver(undefined)).toBeNull() + }) + + it('does not throw on a stray bigint in the envelope (defensive)', () => { + expect(v3ToDriver({ v: 1n } as never)).toBe('{"v":"1"}') + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/column.test.ts b/packages/stack/__tests__/drizzle-v3/column.test.ts new file mode 100644 index 000000000..465938c03 --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/column.test.ts @@ -0,0 +1,136 @@ +import { pgTable } from 'drizzle-orm/pg-core' +import { describe, expect, it } from 'vitest' +import { types as v3Types } from '@/eql/v3' +import { + EQL_V3_DOMAINS, + getEqlV3Column, + isEqlV3Column, + makeEqlV3Column, +} from '@/eql/v3/drizzle/column' +import { + EQL_V3_DOMAIN_SCHEMA, + eqlTypeSlug as slug, + typedEntries, + V3_MATRIX, +} from '../v3-matrix/catalog' + +describe('makeEqlV3Column', () => { + it('sets dataType() to the concrete eql_v3 domain', () => { + const col = makeEqlV3Column(v3Types.IntegerOrd('age')) + const table = pgTable('users', { age: col }) + + expect(table.age.getSQLType()).toBe('public.integer_ord') + }) + + it('recovers the stashed builder before and after pgTable processing', () => { + const col = makeEqlV3Column(v3Types.TextEq('nickname')) + expect(isEqlV3Column(col)).toBe(true) + expect(getEqlV3Column('nickname', col)?.getEqlType()).toBe('public.text_eq') + + const t = pgTable('users', { nickname: col }) + expect(getEqlV3Column('nickname', t.nickname)?.getEqlType()).toBe( + 'public.text_eq', + ) + }) + + it('stashes the builder under a single symbol key, with no string twin', () => { + const symbol = Symbol.for('cipherstash:eqlv3Column') + const carrierOf = (column: unknown) => + (column as { config?: { customTypeParams?: Record } }) + .config?.customTypeParams + + const col = makeEqlV3Column(v3Types.TextEq('nickname')) + const table = pgTable('users', { nickname: col }) + + // `pgTable` builds a NEW PgColumn that does not carry the symbol directly — + // it reaches the builder only through the shared `config.customTypeParams`. + // That is the carrier the operators actually resolve through, so assert on + // it specifically rather than on "some carrier somewhere". + const pgCarrier = carrierOf(table.nickname) + expect(pgCarrier).toBeDefined() + expect(symbol in (pgCarrier as object)).toBe(true) + expect(getEqlV3Column('nickname', table.nickname)?.getEqlType()).toBe( + 'public.text_eq', + ) + + // ...and no carrier keeps a redundant string twin of the symbol key. + for (const carrier of [col, carrierOf(col), table.nickname]) { + expect(carrier).toBeDefined() + expect('_eqlv3Column' in (carrier as object)).toBe(false) + } + }) + + it('every matrix domain slugs to a bare, dot-free column identifier', () => { + // Guards the shared slug against drifting from the real domain schema: if + // the domains move, the prefix constant must move with them or the slug + // silently leaks a qualified name into the generated DDL. + for (const [eqlType] of typedEntries(V3_MATRIX)) { + expect(eqlType.startsWith(`${EQL_V3_DOMAIN_SCHEMA}.`)).toBe(true) + expect(slug(eqlType)).not.toContain('.') + } + }) + + it('EQL_V3_DOMAINS contains every concrete domain', () => { + const all = Object.values(v3Types).map((f) => f('x').getEqlType()) + for (const domain of all) expect(EQL_V3_DOMAINS.has(domain)).toBe(true) + }) + + it('isEqlV3Column is false for a plain value', () => { + expect(isEqlV3Column({})).toBe(false) + }) + + it('recognises v3 columns by the Drizzle getSQLType API', () => { + expect( + isEqlV3Column({ + getSQLType: () => 'public.text_eq', + }), + ).toBe(true) + }) + + it('recovers a v3 builder for columns recognised by getSQLType', () => { + const builder = getEqlV3Column('nickname', { + getSQLType: () => 'public.text_eq', + }) + + expect(builder?.getName()).toBe('nickname') + expect(builder?.getEqlType()).toBe('public.text_eq') + }) + + it('recognises v3 columns by dataType() when getSQLType is absent', () => { + const column = { dataType: () => 'public.text_eq' } + const builder = getEqlV3Column('nickname', column) + + expect(isEqlV3Column(column)).toBe(true) + expect(builder?.getName()).toBe('nickname') + expect(builder?.getEqlType()).toBe('public.text_eq') + }) + + it('recognises v3 columns by sqlName when getSQLType is absent', () => { + const column = { sqlName: 'public.integer_ord' } + const builder = getEqlV3Column('age', column) + + expect(isEqlV3Column(column)).toBe(true) + expect(builder?.getName()).toBe('age') + expect(builder?.getEqlType()).toBe('public.integer_ord') + }) + + it.each( + typedEntries(V3_MATRIX), + )('%s round-trips through makeEqlV3Column and pgTable', (eqlType, spec) => { + const columnName = slug(eqlType) + const builder = spec.builder(columnName) + const column = makeEqlV3Column(builder) + + expect(builder.getEqlType()).toBe(eqlType) + expect(builder).toBeInstanceOf(spec.ColumnClass) + expect(isEqlV3Column(column)).toBe(true) + expect(getEqlV3Column(columnName, column)?.getEqlType()).toBe(eqlType) + + const table = pgTable('users', { [columnName]: column } as never) + const pgColumn = (table as Record)[ + columnName + ] + expect(pgColumn?.getSQLType()).toBe(eqlType) + expect(getEqlV3Column(columnName, pgColumn)?.getEqlType()).toBe(eqlType) + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/exports.test.ts b/packages/stack/__tests__/drizzle-v3/exports.test.ts new file mode 100644 index 000000000..541c06fa8 --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/exports.test.ts @@ -0,0 +1,12 @@ +import { describe, expect, it } from 'vitest' +import * as barrel from '@/eql/v3/drizzle' + +describe('v3 drizzle barrel', () => { + it('exports the public surface', () => { + expect(typeof barrel.createEncryptionOperatorsV3).toBe('function') + expect(typeof barrel.extractEncryptionSchemaV3).toBe('function') + expect(typeof barrel.EncryptionOperatorError).toBe('function') + expect(typeof barrel.types.TextSearch).toBe('function') + expect(barrel.types.IntegerOrd('age')).toBeDefined() + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/operators-live-pg.test.ts b/packages/stack/__tests__/drizzle-v3/operators-live-pg.test.ts new file mode 100644 index 000000000..9819d955b --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/operators-live-pg.test.ts @@ -0,0 +1,687 @@ +import 'dotenv/config' +import { + and, + asc as drizzleAsc, + eq as drizzleEq, + type SQL, + type SQLWrapper, +} from 'drizzle-orm' +import { integer, pgTable, text } from 'drizzle-orm/pg-core' +import { drizzle } from 'drizzle-orm/postgres-js' +import postgres from 'postgres' +import { afterAll, beforeAll, expect, it } from 'vitest' +import { EncryptionV3 } from '@/encryption/v3' +import { + createEncryptionOperatorsV3, + EncryptionOperatorError, + extractEncryptionSchemaV3, + types as v3drizzle, +} from '@/eql/v3/drizzle' +import { makeEqlV3Column } from '@/eql/v3/drizzle/column' +import { installEqlV3IfNeeded } from '../helpers/eql-v3' +import { describeLivePg, LIVE_EQL_V3_PG_ENABLED } from '../helpers/live-gate' +import { + type DomainSpec, + type EqlV3TypeName, + eqlTypeSlug as slug, + typedEntries, + V3_MATRIX, +} from '../v3-matrix/catalog' + +const url = process.env.DATABASE_URL +const sqlClient = LIVE_EQL_V3_PG_ENABLED + ? postgres(url as string, { prepare: false }) + : (undefined as unknown as postgres.Sql) + +const TABLE_NAME = 'protect_ci_v3_drizzle_matrix' +const ACCOUNT_TABLE_NAME = 'protect_ci_v3_drizzle_matrix_accounts' +const RUN = `run-${Date.now()}-${Math.random().toString(36).slice(2, 8)}` +const ROW_A = 'row-a' +const ROW_B = 'row-b' + +const matrixEntries = typedEntries(V3_MATRIX) +const matrixColumns = Object.fromEntries( + matrixEntries.map(([eqlType, spec]) => [ + slug(eqlType), + makeEqlV3Column(spec.builder(slug(eqlType))), + ]), +) + +const matrixTable = pgTable(TABLE_NAME, { + id: integer('id').primaryKey().generatedAlwaysAsIdentity(), + rowKey: text('row_key').notNull(), + testRunId: text('test_run_id').notNull(), + nullableTextEq: makeEqlV3Column( + V3_MATRIX['public.text_eq'].builder('nullable_text_eq'), + ), + ...matrixColumns, +}) + +const accountsTable = pgTable(ACCOUNT_TABLE_NAME, { + id: integer('id').primaryKey().generatedAlwaysAsIdentity(), + rowKey: text('row_key').notNull(), + label: text('label').notNull(), + testRunId: text('test_run_id').notNull(), +}) + +// A statically-typed encrypted table (via the drizzle `types` namespace) with +// concrete bigint columns. Unlike the dynamic matrix table, its column set is +// known at compile time, so it exercises A3 end-to-end with ZERO casts: the +// insert takes envelope rows and the select yields `Encrypted` values ready for +// decrypt. +const BIGINT_TABLE_NAME = 'protect_ci_v3_drizzle_bigint' +const bigintTable = pgTable(BIGINT_TABLE_NAME, { + id: integer('id').primaryKey().generatedAlwaysAsIdentity(), + rowKey: text('row_key').notNull(), + testRunId: text('test_run_id').notNull(), + balance: v3drizzle.BigintOrd('balance'), + ledger: v3drizzle.BigintEq('ledger'), +}) + +// Full i64 bounds — proves protect-ffi 0.28 round-trips a JS bigint beyond +// Number.MAX_SAFE_INTEGER losslessly through the encrypted column. +const BIGINT_BALANCE = 9223372036854775807n +const BIGINT_LEDGER = -9223372036854775808n + +const schema = extractEncryptionSchemaV3(matrixTable) +const bigintSchema = extractEncryptionSchemaV3(bigintTable) + +type PlainValue = string | number | bigint | boolean | Date +type RowKey = typeof ROW_A | typeof ROW_B +type MatrixPlainRow = Record +type SelectRow = { rowKey: string } +type Db = ReturnType +type Client = Awaited> +type Ops = ReturnType +type ComparisonOperator = 'gt' | 'gte' | 'lt' | 'lte' + +let client: Client +let ops: Ops +let db: Db + +const equalityDomains = matrixEntries.filter( + ([, spec]) => spec.indexes.unique || spec.indexes.ore, +) +const orderDomains = matrixEntries.filter(([, spec]) => spec.indexes.ore) +const matchDomains = matrixEntries.filter(([, spec]) => spec.indexes.match) +const noEqualityDomains = matrixEntries.filter( + ([, spec]) => !spec.indexes.unique && !spec.indexes.ore, +) +const noOrderDomains = matrixEntries.filter(([, spec]) => !spec.indexes.ore) +const noMatchDomains = matrixEntries.filter(([, spec]) => !spec.indexes.match) +const comparisonOperators: Array< + [ComparisonOperator, (cmp: number) => boolean] +> = [ + ['gt', (cmp) => cmp > 0], + ['gte', (cmp) => cmp >= 0], + ['lt', (cmp) => cmp < 0], + ['lte', (cmp) => cmp <= 0], +] +const comparisonDomains = orderDomains.flatMap(([eqlType, spec]) => + comparisonOperators.map( + ([operator, predicate]) => [eqlType, spec, operator, predicate] as const, + ), +) + +const matrixColumn = (eqlType: EqlV3TypeName): SQLWrapper => + (matrixTable as unknown as Record)[slug(eqlType)] + +const scoped = (cond: SQL | undefined): SQL | undefined => + cond ? and(drizzleEq(matrixTable.testRunId, RUN), cond) : cond + +const plainValue = (spec: DomainSpec, rowKey: RowKey): PlainValue => + spec.samples[rowKey === ROW_A ? 0 : 1] + +function comparePlain(left: PlainValue, right: PlainValue): number { + if (left instanceof Date && right instanceof Date) { + return left.getTime() - right.getTime() + } + if (typeof left === 'number' && typeof right === 'number') { + return left - right + } + // bigint order domains (`bigint_ord`/`bigint_ord_ore`) carry i64 samples + // beyond Number.MAX_SAFE_INTEGER, so they must be compared as bigints — the + // subtraction is narrowed to -1/0/1 because callers expect a `number`. + if (typeof left === 'bigint' && typeof right === 'bigint') { + return left < right ? -1 : left > right ? 1 : 0 + } + if (typeof left === 'string' && typeof right === 'string') { + // eql_v3 text ordering (ORE) is BYTEWISE, not locale-collated: the oracle + // must model codepoint order, not `localeCompare` (which folds case, + // reorders punctuation vs letters, and is locale-dependent). Text samples + // must stay ASCII/unambiguous so UTF-16 code-unit order == the byte order + // the DB actually sorts by. + return left < right ? -1 : left > right ? 1 : 0 + } + throw new Error( + `Unsupported ordered values: ${String(left)}, ${String(right)}`, + ) +} + +function expectedKeysFor( + spec: DomainSpec, + predicate: (value: PlainValue) => boolean, +): RowKey[] { + return ([ROW_A, ROW_B] as const).filter((rowKey) => + predicate(plainValue(spec, rowKey)), + ) +} + +function sortedKeysFor(spec: DomainSpec, direction: 'asc' | 'desc'): RowKey[] { + return ([ROW_A, ROW_B] as const).sort((left, right) => { + const cmp = comparePlain(plainValue(spec, left), plainValue(spec, right)) + return direction === 'asc' ? cmp : -cmp + }) +} + +async function selectRowKeys(condition: SQL | undefined): Promise { + if (!condition) throw new Error('Expected query condition') + const rows = (await db + .select({ rowKey: matrixTable.rowKey }) + .from(matrixTable) + .where(scoped(condition)) + .orderBy(drizzleAsc(matrixTable.rowKey))) as SelectRow[] + return rows.map((row) => row.rowKey) +} + +function unwrap(result: { data?: T; failure?: { message: string } }): T { + if (result.failure) throw new Error(result.failure.message) + return result.data as T +} + +function encryptedInsertRows(): MatrixPlainRow[] { + return ([ROW_A, ROW_B] as const).map((rowKey) => { + const row: MatrixPlainRow = { + rowKey, + testRunId: RUN, + nullableTextEq: rowKey === ROW_A ? null : 'nullable-present', + } + for (const [eqlType, spec] of matrixEntries) { + row[slug(eqlType)] = plainValue(spec, rowKey) + } + return row + }) +} + +beforeAll(async () => { + if (!LIVE_EQL_V3_PG_ENABLED) return + await installEqlV3IfNeeded(sqlClient) + client = await EncryptionV3({ schemas: [schema, bigintSchema] }) + ops = createEncryptionOperatorsV3(client) + db = drizzle({ client: sqlClient }) + + const columnDefs = matrixEntries + .map(([eqlType]) => `"${slug(eqlType)}" ${eqlType} NOT NULL`) + .join(',\n ') + + await sqlClient.unsafe(` + CREATE TABLE IF NOT EXISTS ${TABLE_NAME} ( + id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, + row_key TEXT NOT NULL, + test_run_id TEXT NOT NULL, + nullable_text_eq public.text_eq, + ${columnDefs} + ) + `) + await sqlClient.unsafe(` + CREATE TABLE IF NOT EXISTS ${ACCOUNT_TABLE_NAME} ( + id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, + row_key TEXT NOT NULL, + label TEXT NOT NULL, + test_run_id TEXT NOT NULL + ) + `) + await sqlClient.unsafe(` + CREATE TABLE IF NOT EXISTS ${BIGINT_TABLE_NAME} ( + id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, + row_key TEXT NOT NULL, + test_run_id TEXT NOT NULL, + balance public.bigint_ord NOT NULL, + ledger public.bigint_eq NOT NULL + ) + `) + + const encryptedRows = unwrap( + await client.bulkEncryptModels(encryptedInsertRows(), schema), + ) + await db.insert(matrixTable).values(encryptedRows) + await db.insert(accountsTable).values([ + { rowKey: ROW_A, label: 'primary', testRunId: RUN }, + { rowKey: ROW_B, label: 'secondary', testRunId: RUN }, + ]) + + // A3 end-to-end, cast-free: encrypt a native bigint model, insert the + // resulting envelope rows (typed against the column's `Encrypted` data slot), + // no `as never` anywhere. + const bigintRows = unwrap( + await client.bulkEncryptModels( + [ + { + rowKey: ROW_A, + testRunId: RUN, + balance: BIGINT_BALANCE, + ledger: BIGINT_LEDGER, + }, + ], + bigintSchema, + ), + ) + await db.insert(bigintTable).values(bigintRows) +}, 120000) + +afterAll(async () => { + if (!LIVE_EQL_V3_PG_ENABLED) return + await sqlClient`DELETE FROM ${sqlClient(TABLE_NAME)} WHERE test_run_id = ${RUN}` + await sqlClient`DELETE FROM ${sqlClient(ACCOUNT_TABLE_NAME)} WHERE test_run_id = ${RUN}` + await sqlClient`DELETE FROM ${sqlClient(BIGINT_TABLE_NAME)} WHERE test_run_id = ${RUN}` + await sqlClient.end() +}, 30000) + +describeLivePg('v3 drizzle operators (live pg matrix)', () => { + it.each(equalityDomains)( + '%s eq selects the exact row', + async (eqlType, spec) => { + const rows = await selectRowKeys( + await ops.eq(matrixColumn(eqlType), plainValue(spec, ROW_A)), + ) + expect(rows).toEqual([ROW_A]) + }, + 30000, + ) + + it.each(equalityDomains)( + '%s ne selects the complement row', + async (eqlType, spec) => { + const rows = await selectRowKeys( + await ops.ne(matrixColumn(eqlType), plainValue(spec, ROW_A)), + ) + expect(rows).toEqual([ROW_B]) + }, + 30000, + ) + + it.each(equalityDomains)( + '%s inArray selects all listed rows', + async (eqlType, spec) => { + const rows = await selectRowKeys( + await ops.inArray(matrixColumn(eqlType), [ + plainValue(spec, ROW_A), + plainValue(spec, ROW_B), + ]), + ) + expect(rows).toEqual([ROW_A, ROW_B]) + }, + 30000, + ) + + it.each(equalityDomains)( + '%s notInArray excludes listed rows', + async (eqlType, spec) => { + const rows = await selectRowKeys( + await ops.notInArray(matrixColumn(eqlType), [plainValue(spec, ROW_A)]), + ) + expect(rows).toEqual([ROW_B]) + }, + 30000, + ) + + it.each(comparisonDomains)( + '%s %s selects rows by encrypted ordering', + async (eqlType, spec, operator, predicate) => { + const bound = plainValue(spec, ROW_A) + const rows = await selectRowKeys( + await ops[operator](matrixColumn(eqlType), bound), + ) + expect(rows).toEqual( + expectedKeysFor(spec, (value) => predicate(comparePlain(value, bound))), + ) + }, + 30000, + ) + + it.each(orderDomains)( + '%s between selects the inclusive range', + async (eqlType, spec) => { + const sortedValues = [ + plainValue(spec, ROW_A), + plainValue(spec, ROW_B), + ].sort(comparePlain) + const rows = await selectRowKeys( + await ops.between( + matrixColumn(eqlType), + sortedValues[0], + sortedValues[1], + ), + ) + expect(rows).toEqual([ROW_A, ROW_B]) + }, + 30000, + ) + + it.each(orderDomains)( + '%s notBetween excludes the inclusive range', + async (eqlType, spec) => { + const sortedValues = [ + plainValue(spec, ROW_A), + plainValue(spec, ROW_B), + ].sort(comparePlain) + const rows = await selectRowKeys( + await ops.notBetween( + matrixColumn(eqlType), + sortedValues[0], + sortedValues[1], + ), + ) + expect(rows).toEqual([]) + }, + 30000, + ) + + // The spanning cases above only ever prove INCLUSION (both rows are inside + // the range, so `between` -> [A,B] and `notBetween` -> []). These narrow + // cases use a single-point range at ROW_A's value to prove the operators + // also EXCLUDE: `between` must drop ROW_B, `notBetween` must keep it. Without + // these, a `between` that matched everything (or a `notBetween` no-op) passes. + it.each(orderDomains)( + '%s between at a single point excludes the out-of-range row', + async (eqlType, spec) => { + const bound = plainValue(spec, ROW_A) + const rows = await selectRowKeys( + await ops.between(matrixColumn(eqlType), bound, bound), + ) + expect(rows).toEqual( + expectedKeysFor(spec, (value) => comparePlain(value, bound) === 0), + ) + }, + 30000, + ) + + it.each(orderDomains)( + '%s notBetween at a single point keeps the out-of-range row', + async (eqlType, spec) => { + const bound = plainValue(spec, ROW_A) + const rows = await selectRowKeys( + await ops.notBetween(matrixColumn(eqlType), bound, bound), + ) + expect(rows).toEqual( + expectedKeysFor(spec, (value) => comparePlain(value, bound) !== 0), + ) + }, + 30000, + ) + + // `between` returns a two-function conjunction; Drizzle's passthrough `not` + // prepends a bare `not` with no parentheses of its own. Postgres binds NOT + // tighter than AND, so an unparenthesised range would run as + // `(NOT gte(col, bound)) AND lte(col, bound)` — i.e. `col < bound` — instead + // of the range's complement, `col != bound`. + // + // The bound MUST be the SMALLER of the two seeded values. Anchored at the + // larger one, `col < bound` and `col != bound` both select the other row and + // the buggy SQL passes; anchored at the smaller, the buggy SQL selects + // nothing while the correct SQL selects the larger row. Several domains seed + // ROW_A with their maximum sample (`integer_ord` is `[0, -42]`), so keying + // off ROW_A directly would make this test vacuous for them. + it.each(orderDomains)( + '%s not(between(...)) negates the whole range', + async (eqlType, spec) => { + const bound = [plainValue(spec, ROW_A), plainValue(spec, ROW_B)].sort( + comparePlain, + )[0] + const rows = await selectRowKeys( + ops.not(await ops.between(matrixColumn(eqlType), bound, bound)), + ) + expect(rows).toEqual( + expectedKeysFor(spec, (value) => comparePlain(value, bound) !== 0), + ) + // Guards the guard: the expectation must be non-empty, or the buggy + // `col < bound` (which returns nothing) would satisfy it too. + expect(rows.length).toBeGreaterThan(0) + }, + 30000, + ) + + it.each(orderDomains)( + '%s asc orders by encrypted order term', + async (eqlType, spec) => { + const rows = (await db + .select({ rowKey: matrixTable.rowKey }) + .from(matrixTable) + .where(drizzleEq(matrixTable.testRunId, RUN)) + .orderBy(ops.asc(matrixColumn(eqlType)))) as SelectRow[] + expect(rows.map((row) => row.rowKey)).toEqual(sortedKeysFor(spec, 'asc')) + }, + 30000, + ) + + it.each(orderDomains)( + '%s desc orders by encrypted order term', + async (eqlType, spec) => { + const rows = (await db + .select({ rowKey: matrixTable.rowKey }) + .from(matrixTable) + .where(drizzleEq(matrixTable.testRunId, RUN)) + .orderBy(ops.desc(matrixColumn(eqlType)))) as SelectRow[] + expect(rows.map((row) => row.rowKey)).toEqual(sortedKeysFor(spec, 'desc')) + }, + 30000, + ) + + it.each(matchDomains)( + '%s contains matches plaintext terms', + async (eqlType, spec) => { + const rows = await selectRowKeys( + await ops.contains(matrixColumn(eqlType), 'ada'), + ) + expect(rows).toEqual( + expectedKeysFor(spec, (value) => + String(value).toLowerCase().includes('ada'), + ), + ) + }, + 30000, + ) + + it.each( + noEqualityDomains, + )('%s eq rejects unsupported equality', async (eqlType, spec) => { + await expect( + ops.eq(matrixColumn(eqlType), plainValue(spec, ROW_A)), + ).rejects.toBeInstanceOf(EncryptionOperatorError) + }) + + it.each( + noOrderDomains, + )('%s gt rejects unsupported ordering', async (eqlType, spec) => { + await expect( + ops.gt(matrixColumn(eqlType), plainValue(spec, ROW_A)), + ).rejects.toBeInstanceOf(EncryptionOperatorError) + }) + + it.each(noOrderDomains)('%s asc rejects unsupported ordering', (eqlType) => { + expect(() => ops.asc(matrixColumn(eqlType))).toThrow( + EncryptionOperatorError, + ) + }) + + it.each( + noMatchDomains, + )('%s contains rejects unsupported match', async (eqlType, spec) => { + await expect( + ops.contains(matrixColumn(eqlType), plainValue(spec, ROW_A)), + ).rejects.toBeInstanceOf(EncryptionOperatorError) + }) + + it('and combines encrypted predicates', async () => { + const rows = await selectRowKeys( + await ops.and( + ops.eq(matrixColumn('public.text_eq'), 'ada@example.com'), + ops.lt(matrixColumn('public.integer_ord'), 0), + ), + ) + expect(rows).toEqual([ROW_B]) + }, 30000) + + it('or combines encrypted predicates', async () => { + const rows = await selectRowKeys( + await ops.or( + ops.eq(matrixColumn('public.text_eq'), ''), + ops.eq(matrixColumn('public.text_eq'), 'ada@example.com'), + ), + ) + expect(rows).toEqual([ROW_A, ROW_B]) + }, 30000) + + it('not negates an encrypted predicate', async () => { + const rows = await selectRowKeys( + ops.not(await ops.eq(matrixColumn('public.text_eq'), '')), + ) + expect(rows).toEqual([ROW_B]) + }, 30000) + + it('isNull and isNotNull work on nullable encrypted columns', async () => { + expect(await selectRowKeys(ops.isNull(matrixTable.nullableTextEq))).toEqual( + [ROW_A], + ) + expect( + await selectRowKeys(ops.isNotNull(matrixTable.nullableTextEq)), + ).toEqual([ROW_B]) + }, 30000) + + it('exists and notExists work with correlated subqueries', async () => { + const primaryAccount = db + .select({ id: accountsTable.id }) + .from(accountsTable) + .where( + and( + drizzleEq(accountsTable.testRunId, RUN), + drizzleEq(accountsTable.rowKey, matrixTable.rowKey), + drizzleEq(accountsTable.label, 'primary'), + ), + ) + const missingAccount = db + .select({ id: accountsTable.id }) + .from(accountsTable) + .where( + and( + drizzleEq(accountsTable.testRunId, RUN), + drizzleEq(accountsTable.rowKey, matrixTable.rowKey), + drizzleEq(accountsTable.label, 'missing'), + ), + ) + + expect(await selectRowKeys(ops.exists(primaryAccount))).toEqual([ROW_A]) + expect(await selectRowKeys(ops.notExists(missingAccount))).toEqual([ + ROW_A, + ROW_B, + ]) + }, 30000) + + it('joins plain tables while filtering encrypted columns', async () => { + const rows = (await db + .select({ rowKey: matrixTable.rowKey }) + .from(matrixTable) + .innerJoin( + accountsTable, + and( + drizzleEq(accountsTable.testRunId, RUN), + drizzleEq(accountsTable.rowKey, matrixTable.rowKey), + ), + ) + .where( + scoped(await ops.eq(matrixColumn('public.text_eq'), 'ada@example.com')), + )) as SelectRow[] + expect(rows.map((row) => row.rowKey)).toEqual([ROW_B]) + }, 30000) + + it('paginates encrypted ordering results with limit and offset', async () => { + const spec = V3_MATRIX['public.integer_ord'] + const rows = (await db + .select({ rowKey: matrixTable.rowKey }) + .from(matrixTable) + .where(drizzleEq(matrixTable.testRunId, RUN)) + .orderBy(ops.asc(matrixColumn('public.integer_ord'))) + .limit(1) + .offset(1)) as SelectRow[] + expect(rows.map((row) => row.rowKey)).toEqual( + sortedKeysFor(spec, 'asc').slice(1, 2), + ) + }, 30000) + + // A real `TypedEncryptionClient` exposes `bulkEncrypt`, so these lists are + // encrypted in one FFI crossing and the returned terms must line up + // index-for-index with the values. Five values also crosses the + // MAX_IN_ARRAY_CONCURRENCY=4 boundary of the single-encrypt fallback that a + // `bulkEncrypt`-less client would take. Either way the OR/AND of eq/ne terms + // must be correct — a misaligned bulk response would silently select the + // wrong rows here. + it('inArray encrypts a >4-value list in one bulk crossing', async () => { + const rows = await selectRowKeys( + await ops.inArray(matrixColumn('public.text_eq'), [ + 'ada@example.com', + '', + 'nobody-1@example.com', + 'nobody-2@example.com', + 'nobody-3@example.com', + ]), + ) + // '' -> ROW_A, 'ada@example.com' -> ROW_B; the three "nobody" terms match + // nothing, exercising the pool without changing the expected set. + expect(rows).toEqual([ROW_A, ROW_B]) + }, 30000) + + it('notInArray encrypts a >4-value list in one bulk crossing', async () => { + const rows = await selectRowKeys( + await ops.notInArray(matrixColumn('public.text_eq'), [ + '', + 'nobody-1@example.com', + 'nobody-2@example.com', + 'nobody-3@example.com', + 'nobody-4@example.com', + ]), + ) + // Only '' is excluded (ROW_A); ROW_B ('ada@example.com') survives. + expect(rows).toEqual([ROW_B]) + }, 30000) + + // A3 + bigint lock: a statically-typed bigint table round-trips a real i64 + // value through encrypt → insert → select → decrypt with no casts. The select + // yields `Encrypted`-typed columns (the envelope), fed straight to decrypt. + it('round-trips a native bigint through a statically-typed encrypted column', async () => { + const encrypted = await db + .select({ balance: bigintTable.balance, ledger: bigintTable.ledger }) + .from(bigintTable) + .where(drizzleEq(bigintTable.testRunId, RUN)) + expect(encrypted).toHaveLength(1) + + const decrypted = unwrap( + await client.decryptModel(encrypted[0], bigintSchema), + ) + expect(decrypted.balance).toBe(BIGINT_BALANCE) + expect(decrypted.ledger).toBe(BIGINT_LEDGER) + }, 30000) + + it('filters a bigint column by encrypted equality and ordering', async () => { + const byLedger = (await db + .select({ rowKey: bigintTable.rowKey }) + .from(bigintTable) + .where( + and( + drizzleEq(bigintTable.testRunId, RUN), + await ops.eq(bigintTable.ledger, BIGINT_LEDGER), + ), + )) as SelectRow[] + expect(byLedger.map((row) => row.rowKey)).toEqual([ROW_A]) + + const byBalance = (await db + .select({ rowKey: bigintTable.rowKey }) + .from(bigintTable) + .where( + and( + drizzleEq(bigintTable.testRunId, RUN), + await ops.gt(bigintTable.balance, 0n), + ), + )) as SelectRow[] + expect(byBalance.map((row) => row.rowKey)).toEqual([ROW_A]) + }, 30000) +}) diff --git a/packages/stack/__tests__/drizzle-v3/operators-lock-context-live-pg.test.ts b/packages/stack/__tests__/drizzle-v3/operators-lock-context-live-pg.test.ts new file mode 100644 index 000000000..94487619a --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/operators-lock-context-live-pg.test.ts @@ -0,0 +1,181 @@ +/** + * Live, identity-bound querying through the v3 Drizzle operator path. + * + * `matrix-identity-live.test.ts` proves lock context round-trips through the + * typed CLIENT (`encryptModel`/`decryptModel`), and `operators.test.ts` proves + * the Drizzle operators FORWARD `lockContext`/`audit` — but only against a + * MOCKED FFI. Nothing exercises the one end-to-end shape that matters most: + * seed rows bound to an identity, then query them with `ops.eq(col, value, + * { lockContext })` against a real database and assert the encrypted term + * actually matches the stored row and decrypts. + * + * Wiring mirrors `lock-context.test.ts` (the current, non-deprecated + * strategy-based flow): the client authenticates as the end user via + * `OidcFederationStrategy` and binds the key to the `sub` claim with a plain + * `.withLockContext({ identityClaim })`. + * + * Gated twice: `describeLivePg` (needs `DATABASE_URL` + CS creds) and an inner + * `USER_JWT` guard (soft-skip, matching the existing identity/lock-context + * suites). Whether the searchable index terms are themselves identity-bound is + * decided inside `@cipherstash/protect-ffi`, not this repo — so we assert the + * SYMMETRIC behaviour (same lock context on seed + query matches and decrypts), + * not a cross-identity non-match. + */ +import 'dotenv/config' +import { OidcFederationStrategy } from '@cipherstash/auth' +import { and, asc as drizzleAsc, eq as drizzleEq, type SQL } from 'drizzle-orm' +import { integer, pgTable, text } from 'drizzle-orm/pg-core' +import { drizzle } from 'drizzle-orm/postgres-js' +import postgres from 'postgres' +import { afterAll, beforeAll, expect, it } from 'vitest' +import { EncryptionV3 } from '@/encryption/v3' +import { + createEncryptionOperatorsV3, + extractEncryptionSchemaV3, +} from '@/eql/v3/drizzle' +import { makeEqlV3Column } from '@/eql/v3/drizzle/column' +import { installEqlV3IfNeeded } from '../helpers/eql-v3' +import { describeLivePg, LIVE_EQL_V3_PG_ENABLED } from '../helpers/live-gate' +import { V3_MATRIX } from '../v3-matrix/catalog' + +const url = process.env.DATABASE_URL +const sqlClient = LIVE_EQL_V3_PG_ENABLED + ? postgres(url as string, { prepare: false }) + : (undefined as unknown as postgres.Sql) + +const TABLE_NAME = 'protect_ci_v3_drizzle_lock_context' +const RUN = `run-${Date.now()}-${Math.random().toString(36).slice(2, 8)}` +const ROW_A = 'row-a' +const ROW_B = 'row-b' +const SECRET_A = 'ada@example.com' +const SECRET_B = 'grace@example.com' + +// A fixed identity claim; the same value must be supplied on encrypt and query +// for the terms/keys to reproduce. +const IDENTITY_CLAIM = { identityClaim: ['sub'] } + +const secretTable = pgTable(TABLE_NAME, { + id: integer('id').primaryKey().generatedAlwaysAsIdentity(), + rowKey: text('row_key').notNull(), + testRunId: text('test_run_id').notNull(), + secret: makeEqlV3Column(V3_MATRIX['public.text_eq'].builder('secret')), +} as never) + +const schema = extractEncryptionSchemaV3(secretTable) + +type SelectRow = { rowKey: string } + +let client: Awaited> +let ops: ReturnType +let db: ReturnType +let userJwt: string | undefined + +function unwrap(result: { data?: T; failure?: { message: string } }): T { + if (result.failure) throw new Error(result.failure.message) + return result.data as T +} + +/** Run-scoped SELECT of row keys under an already-encrypted SQL condition. */ +async function selectRowKeys(condition: SQL): Promise { + const rows = (await db + .select({ rowKey: secretTable.rowKey }) + .from(secretTable) + .where(and(drizzleEq(secretTable.testRunId, RUN), condition)) + .orderBy(drizzleAsc(secretTable.rowKey))) as SelectRow[] + return rows.map((row) => row.rowKey) +} + +beforeAll(async () => { + if (!LIVE_EQL_V3_PG_ENABLED) return + userJwt = process.env.USER_JWT + if (!userJwt) return + + const crn = process.env.CS_WORKSPACE_CRN + if (!crn) + throw new Error('CS_WORKSPACE_CRN must be set for lock-context tests') + + await installEqlV3IfNeeded(sqlClient) + client = await EncryptionV3({ + schemas: [schema], + config: { + strategy: OidcFederationStrategy.create(crn, () => + Promise.resolve(userJwt as string), + ), + }, + }) + ops = createEncryptionOperatorsV3(client) + db = drizzle({ client: sqlClient }) + + await sqlClient.unsafe(` + CREATE TABLE IF NOT EXISTS ${TABLE_NAME} ( + id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, + row_key TEXT NOT NULL, + test_run_id TEXT NOT NULL, + secret public.text_eq NOT NULL + ) + `) + + // Seed BOTH rows bound to the same lock context. + const encryptedRows = unwrap>>( + await client + .bulkEncryptModels( + [ + { rowKey: ROW_A, testRunId: RUN, secret: SECRET_A }, + { rowKey: ROW_B, testRunId: RUN, secret: SECRET_B }, + ] as never, + schema, + ) + .withLockContext(IDENTITY_CLAIM), + ) + await db.insert(secretTable).values(encryptedRows as never) +}, 120000) + +afterAll(async () => { + if (!LIVE_EQL_V3_PG_ENABLED) return + if (userJwt) { + await sqlClient`DELETE FROM ${sqlClient(TABLE_NAME)} WHERE test_run_id = ${RUN}` + } + await sqlClient.end() +}, 30000) + +describeLivePg('v3 drizzle operators with lock context (live pg)', () => { + const skipUnlessJwt = (): boolean => { + if (!userJwt) { + console.log('Skipping lock-context operator test - no USER_JWT provided') + return true + } + return false + } + + it('eq with a matching lock context selects the exact row', async () => { + if (skipUnlessJwt()) return + const condition = await ops.eq(secretTable.secret, SECRET_A, { + // Runtime accepts a plain { identityClaim } (forwarded to + // withLockContext); the operator opts type is narrowed to LockContext. + lockContext: IDENTITY_CLAIM as never, + }) + expect(await selectRowKeys(condition)).toEqual([ROW_A]) + }, 30000) + + it('a lock-context-bound row decrypts with the same lock context', async () => { + if (skipUnlessJwt()) return + const [row] = await sqlClient.unsafe>( + `SELECT secret::jsonb AS value FROM ${TABLE_NAME} + WHERE test_run_id = $1 AND row_key = $2`, + [RUN, ROW_A], + ) + const decrypted = unwrap( + await client.decrypt(row.value as never).withLockContext(IDENTITY_CLAIM), + ) + expect(decrypted).toBe(SECRET_A) + }, 30000) + + it('eq threads an audit config alongside the lock context', async () => { + if (skipUnlessJwt()) return + const condition = await ops.eq(secretTable.secret, SECRET_B, { + lockContext: IDENTITY_CLAIM as never, + audit: { metadata: { sub: 'toby@cipherstash.com', type: 'query' } }, + }) + expect(await selectRowKeys(condition)).toEqual([ROW_B]) + }, 30000) +}) diff --git a/packages/stack/__tests__/drizzle-v3/operators-null-live-pg.test.ts b/packages/stack/__tests__/drizzle-v3/operators-null-live-pg.test.ts new file mode 100644 index 000000000..d224250b6 --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/operators-null-live-pg.test.ts @@ -0,0 +1,177 @@ +/** + * Live NULL persistence for encrypted columns across every capability tier. + * + * `operators-live-pg.test.ts` proves `isNull`/`isNotNull` — but only for a + * single `text_eq` column (`nullableTextEq`). NULL storage and retrieval for + * the other tiers (storage-only, order/ORE, free-text match) is untested live: + * a bug that mangled a NULL cell, or a domain that rejected NULL, would only + * show up on those column kinds. + * + * This file seeds two rows — row A all-NULL, row B all-present — across one + * representative column per tier, then asserts, per column: `isNull` selects + * the NULL row, `isNotNull` selects the present row, the NULL cell reads back + * as SQL NULL, and the present cell still decrypts to its plaintext. + */ +import 'dotenv/config' +import { and, asc as drizzleAsc, eq as drizzleEq, type SQL } from 'drizzle-orm' +import { integer, pgTable, text } from 'drizzle-orm/pg-core' +import { drizzle } from 'drizzle-orm/postgres-js' +import postgres from 'postgres' +import { afterAll, beforeAll, expect, it } from 'vitest' +import { EncryptionV3 } from '@/encryption/v3' +import { + createEncryptionOperatorsV3, + extractEncryptionSchemaV3, +} from '@/eql/v3/drizzle' +import { makeEqlV3Column } from '@/eql/v3/drizzle/column' +import { installEqlV3IfNeeded } from '../helpers/eql-v3' +import { describeLivePg, LIVE_EQL_V3_PG_ENABLED } from '../helpers/live-gate' +import { V3_MATRIX } from '../v3-matrix/catalog' + +const url = process.env.DATABASE_URL +const sqlClient = LIVE_EQL_V3_PG_ENABLED + ? postgres(url as string, { prepare: false }) + : (undefined as unknown as postgres.Sql) + +const TABLE_NAME = 'protect_ci_v3_drizzle_nullable' +const RUN = `run-${Date.now()}-${Math.random().toString(36).slice(2, 8)}` +const ROW_A = 'row-a' // all NULL +const ROW_B = 'row-b' // all present + +// One representative column per capability tier, each NULLABLE. +const nullableTable = pgTable(TABLE_NAME, { + id: integer('id').primaryKey().generatedAlwaysAsIdentity(), + rowKey: text('row_key').notNull(), + testRunId: text('test_run_id').notNull(), + storageText: makeEqlV3Column( + V3_MATRIX['public.text'].builder('storage_text'), + ), + eqText: makeEqlV3Column(V3_MATRIX['public.text_eq'].builder('eq_text')), + ordInt: makeEqlV3Column(V3_MATRIX['public.integer_ord'].builder('ord_int')), + matchText: makeEqlV3Column( + V3_MATRIX['public.text_match'].builder('match_text'), + ), +} as never) + +// Tier metadata: property (drizzle) + DB column + a present-row plaintext. +const TIERS = [ + { + key: 'storageText', + db: 'storage_text', + domain: 'public.text', + sample: 'stored-secret', + }, + { + key: 'eqText', + db: 'eq_text', + domain: 'public.text_eq', + sample: 'ada@example.com', + }, + { key: 'ordInt', db: 'ord_int', domain: 'public.integer_ord', sample: 42 }, + { + key: 'matchText', + db: 'match_text', + domain: 'public.text_match', + sample: 'ada lovelace', + }, +] as const + +const schema = extractEncryptionSchemaV3(nullableTable) + +type SelectRow = { rowKey: string } + +let client: Awaited> +let ops: ReturnType +let db: ReturnType + +function unwrap(result: { data?: T; failure?: { message: string } }): T { + if (result.failure) throw new Error(result.failure.message) + return result.data as T +} + +const columnFor = (key: string): SQL => + (nullableTable as unknown as Record)[key] + +async function selectRowKeys(condition: SQL): Promise { + const rows = (await db + .select({ rowKey: nullableTable.rowKey }) + .from(nullableTable) + .where(and(drizzleEq(nullableTable.testRunId, RUN), condition)) + .orderBy(drizzleAsc(nullableTable.rowKey))) as SelectRow[] + return rows.map((row) => row.rowKey) +} + +beforeAll(async () => { + if (!LIVE_EQL_V3_PG_ENABLED) return + await installEqlV3IfNeeded(sqlClient) + client = await EncryptionV3({ schemas: [schema] }) + ops = createEncryptionOperatorsV3(client) + db = drizzle({ client: sqlClient }) + + const columnDefs = TIERS.map((t) => `"${t.db}" ${t.domain}`).join(',\n ') + await sqlClient.unsafe(` + CREATE TABLE IF NOT EXISTS ${TABLE_NAME} ( + id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, + row_key TEXT NOT NULL, + test_run_id TEXT NOT NULL, + ${columnDefs} + ) + `) + + // Row A: every encrypted column NULL. Row B: every column present. + const rowA: Record = { rowKey: ROW_A, testRunId: RUN } + const rowB: Record = { rowKey: ROW_B, testRunId: RUN } + for (const t of TIERS) { + rowA[t.key] = null + rowB[t.key] = t.sample + } + + const encryptedRows = unwrap>>( + await client.bulkEncryptModels([rowA, rowB] as never, schema), + ) + await db.insert(nullableTable).values(encryptedRows as never) +}, 120000) + +afterAll(async () => { + if (!LIVE_EQL_V3_PG_ENABLED) return + await sqlClient`DELETE FROM ${sqlClient(TABLE_NAME)} WHERE test_run_id = ${RUN}` + await sqlClient.end() +}, 30000) + +describeLivePg('v3 drizzle NULL persistence across tiers (live pg)', () => { + it.each(TIERS)('$domain isNull selects the NULL row', async (tier) => { + expect(await selectRowKeys(ops.isNull(columnFor(tier.key)))).toEqual([ + ROW_A, + ]) + }, 30000) + + it.each(TIERS)('$domain isNotNull selects the present row', async (tier) => { + expect(await selectRowKeys(ops.isNotNull(columnFor(tier.key)))).toEqual([ + ROW_B, + ]) + }, 30000) + + it.each( + TIERS, + )('$domain stores a real NULL for the null row', async (tier) => { + const [row] = await sqlClient.unsafe>( + `SELECT "${tier.db}"::jsonb AS value FROM ${TABLE_NAME} + WHERE test_run_id = $1 AND row_key = $2`, + [RUN, ROW_A], + ) + expect(row.value).toBeNull() + }, 30000) + + it.each( + TIERS, + )('$domain present cell decrypts to its plaintext', async (tier) => { + const [row] = await sqlClient.unsafe>( + `SELECT "${tier.db}"::jsonb AS value FROM ${TABLE_NAME} + WHERE test_run_id = $1 AND row_key = $2`, + [RUN, ROW_B], + ) + expect(row.value).toHaveProperty('c') + const decrypted = unwrap(await client.decrypt(row.value as never)) + expect(decrypted).toBe(tier.sample) + }, 30000) +}) diff --git a/packages/stack/__tests__/drizzle-v3/operators.test-d.ts b/packages/stack/__tests__/drizzle-v3/operators.test-d.ts new file mode 100644 index 000000000..564c6615d --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/operators.test-d.ts @@ -0,0 +1,35 @@ +import { describe, expectTypeOf, it } from 'vitest' +import type { EncryptionClient } from '@/encryption' +import type { EncryptionV3 } from '@/encryption/v3' +import { createEncryptionOperatorsV3 } from '@/eql/v3/drizzle' + +/** + * Static regression guard for M1: `createEncryptionOperatorsV3` must accept the + * `TypedEncryptionClient` that `EncryptionV3` resolves to — the documented + * `createEncryptionOperatorsV3(await EncryptionV3({ schemas }))` usage — as well + * as the nominal `EncryptionClient` and a hand-rolled `{ encrypt }` double, + * none requiring a cast. Typing the parameter to `EncryptionClient` (the + * original bug) makes the first call below a compile error, which this suite + * would then catch. Lives in a `*.test-d.ts` so it is inside the existing + * typecheck scope without dragging the loose-typed runtime suites in. + */ +describe('createEncryptionOperatorsV3 - client parameter (M1)', () => { + type V3Client = Awaited> + + it('accepts the client EncryptionV3 returns with no cast', () => { + expectTypeOf(createEncryptionOperatorsV3).toBeCallableWith({} as V3Client) + }) + + it('still accepts the nominal EncryptionClient', () => { + expectTypeOf(createEncryptionOperatorsV3).toBeCallableWith( + {} as EncryptionClient, + ) + }) + + it('accepts a minimal structural { encrypt } double', () => { + const double = { + encrypt: (_plaintext: never, _opts: never) => ({}) as unknown, + } + expectTypeOf(createEncryptionOperatorsV3).toBeCallableWith(double) + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/operators.test.ts b/packages/stack/__tests__/drizzle-v3/operators.test.ts new file mode 100644 index 000000000..219ecba06 --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/operators.test.ts @@ -0,0 +1,629 @@ +import { + eq as drizzleEq, + exists, + isNotNull, + isNull, + not, + notExists, + type SQL, + type SQLWrapper, + sql, +} from 'drizzle-orm' +import { integer, PgDialect, pgTable } from 'drizzle-orm/pg-core' +import { describe, expect, it, vi } from 'vitest' +import { makeEqlV3Column } from '@/eql/v3/drizzle/column' +import { + createEncryptionOperatorsV3, + EncryptionOperatorError, +} from '@/eql/v3/drizzle/operators' +import { extractEncryptionSchemaV3 } from '@/eql/v3/drizzle/schema-extraction' +import { types } from '@/eql/v3/drizzle/types' +import { + eqlTypeSlug as slug, + typedEntries, + V3_MATRIX, +} from '../v3-matrix/catalog' + +const TERM = { c: 'ct', v: 1 } +const TERM_JSON = JSON.stringify(TERM) +const lockContext = { identityClaim: 'user-123' } +const audit = { metadata: { actor: 'test' } } + +type EncryptResult = Promise< + { data: typeof TERM } | { failure: { message: string } } +> + +function chainable(result: EncryptResult) { + const op = result as EncryptResult & { + withLockContext: ReturnType + audit: ReturnType + } + op.withLockContext = vi.fn(() => op) + op.audit = vi.fn(() => op) + return op +} + +function setup( + encryptImpl: (...args: unknown[]) => EncryptResult = async () => ({ + data: TERM, + }), +) { + const encrypt = vi.fn((...args: unknown[]) => chainable(encryptImpl(...args))) + // The factory's `client` parameter is the structural `{ encrypt }` surface, + // so this hand-rolled double satisfies it with no cast (M1). + const client = { encrypt } + const ops = createEncryptionOperatorsV3(client, { lockContext, audit }) + const dialect = new PgDialect() + const render = (s: unknown) => dialect.sqlToQuery(s as SQL) + return { ops, encrypt, render } +} + +type BulkPayload = Array<{ id?: string; plaintext: unknown }> +type BulkResult = Promise< + { data: Array<{ data: typeof TERM }> } | { failure: { message: string } } +> + +/** + * A double for the fuller client surface — one that also exposes `bulkEncrypt`, + * as the real `TypedEncryptionClient` does. `inArray`/`notInArray` should + * prefer it over N single `encrypt` crossings. + */ +function setupBulk( + bulkImpl: (payloads: BulkPayload) => BulkResult = async (payloads) => ({ + data: payloads.map(() => ({ data: TERM })), + }), +) { + const encrypt = vi.fn(() => chainable(Promise.resolve({ data: TERM }))) + const bulkEncrypt = vi.fn((payloads: BulkPayload, ..._rest: unknown[]) => + chainable(bulkImpl(payloads) as never), + ) + const client = { encrypt, bulkEncrypt } + const ops = createEncryptionOperatorsV3(client, { lockContext, audit }) + const dialect = new PgDialect() + const render = (s: unknown) => dialect.sqlToQuery(s as SQL) + return { ops, encrypt, bulkEncrypt, render } +} + +const matrixEntries = typedEntries(V3_MATRIX) +const matrixTable = pgTable( + 'matrix_users', + Object.fromEntries( + matrixEntries.map(([eqlType, spec]) => [ + slug(eqlType), + makeEqlV3Column(spec.builder(slug(eqlType))), + ]), + ) as never, +) +const matrixColumn = (eqlType: string) => + (matrixTable as Record)[slug(eqlType)] +const sampleFor = (spec: (typeof V3_MATRIX)[keyof typeof V3_MATRIX]) => + spec.samples[0] + +const equalityDomains = matrixEntries.filter( + ([, spec]) => spec.indexes.unique || spec.indexes.ore, +) +const orderDomains = matrixEntries.filter(([, spec]) => spec.indexes.ore) +const matchDomains = matrixEntries.filter(([, spec]) => spec.indexes.match) +const storageDomains = matrixEntries.filter( + ([, spec]) => + !spec.indexes.unique && !spec.indexes.ore && !spec.indexes.match, +) + +const users = pgTable('users', { + id: integer().primaryKey(), + email: types.TextSearch('email'), + nickname: types.TextEq('nickname'), + textMatch: types.TextMatch('text_match'), + textOrd: types.TextOrd('text_ord'), + textOrdOre: types.TextOrdOre('text_ord_ore'), + int2Age: types.SmallintOrd('int2_age'), + age: types.IntegerOrd('age'), + createdOn: types.DateOrd('created_on'), + createdAt: types.TimestampOrd('created_at'), + amount: types.NumericOrd('amount'), + score4: types.RealOrd('score4'), + score8: types.DoubleOrd('score8'), + flag: types.Boolean('flag'), +}) + +describe('createEncryptionOperatorsV3 - equality', () => { + it.each( + equalityDomains, + )('%s eq emits the latest two-arg eql_v3.eq with a full-envelope operand', async (eqlType, spec) => { + const { ops, encrypt, render } = setup() + const q = render(await ops.eq(matrixColumn(eqlType), sampleFor(spec))) + + expect(q.sql).toContain( + `eql_v3.eq("matrix_users"."${slug(eqlType)}", $1::jsonb)`, + ) + expect(q.params).toEqual([TERM_JSON]) + expect(encrypt.mock.calls[0]?.[1]?.column.getName()).toBe(slug(eqlType)) + }) + + it.each(equalityDomains)('%s ne emits eql_v3.neq', async (eqlType, spec) => { + const { ops, encrypt, render } = setup() + const q = render(await ops.ne(matrixColumn(eqlType), sampleFor(spec))) + + expect(q.sql).toContain( + `eql_v3.neq("matrix_users"."${slug(eqlType)}", $1::jsonb)`, + ) + expect(q.params).toEqual([TERM_JSON]) + expect(encrypt.mock.calls[0]?.[1]?.column.getName()).toBe(slug(eqlType)) + }) + + it('same-named columns on different tables use their own equality capability', async () => { + const accounts = pgTable('accounts', { + email: types.TextEq('email'), + }) + pgTable('metrics', { + email: types.IntegerOrd('email'), + }) + const { ops, render } = setup() + + const q = render(await ops.eq(accounts.email, 'ada@example.com')) + + expect(q.sql).toContain('eql_v3.eq("accounts"."email", $1::jsonb)') + }) + + it('does not reuse a cached extracted schema across distinct pgTable objects with the same SQL name', async () => { + const first = pgTable('shared', { + email: types.TextEq('email'), + }) + const second = pgTable('shared', { + age: types.IntegerOrd('age'), + }) + const { ops, encrypt } = setup() + + await ops.eq(first.email, 'ada@example.com') + await ops.eq(second.age, 37) + + expect(encrypt.mock.calls[1]?.[1]?.table.build()).toEqual( + extractEncryptionSchemaV3(second).build(), + ) + }) + + it('passes default lock context and audit to operand encryption', async () => { + const { ops, encrypt } = setup() + await ops.eq(users.nickname, 'ada') + const op = encrypt.mock.results[0]?.value + expect(op.withLockContext).toHaveBeenCalledWith(lockContext) + expect(op.audit).toHaveBeenCalledWith(audit) + }) + + it('per-call lock context and audit override constructor defaults', async () => { + const { ops, encrypt } = setup() + const callLockContext = { identityClaim: 'user-456' } + const callAudit = { metadata: { actor: 'override' } } + + await ops.eq(users.nickname, 'ada', { + lockContext: callLockContext, + audit: callAudit, + }) + + const op = encrypt.mock.results[0]?.value + expect(op.withLockContext).toHaveBeenCalledWith(callLockContext) + expect(op.withLockContext).not.toHaveBeenCalledWith(lockContext) + expect(op.audit).toHaveBeenCalledWith(callAudit) + expect(op.audit).not.toHaveBeenCalledWith(audit) + }) +}) + +describe('createEncryptionOperatorsV3 - comparison & range', () => { + it.each([ + ['gt', 'eql_v3.gt'], + ['gte', 'eql_v3.gte'], + ['lt', 'eql_v3.lt'], + ['lte', 'eql_v3.lte'], + ] as const)('%s emits %s for every ORE domain', async (op, fn) => { + for (const [eqlType, spec] of orderDomains) { + const { ops, encrypt, render } = setup() + const q = render(await ops[op](matrixColumn(eqlType), sampleFor(spec))) + + expect(q.sql).toContain( + `${fn}("matrix_users"."${slug(eqlType)}", $1::jsonb)`, + ) + expect(q.params).toEqual([TERM_JSON]) + expect(encrypt.mock.calls[0]?.[1]?.column.getName()).toBe(slug(eqlType)) + } + }) + + it.each( + orderDomains, + )('%s between emits a bounded range with two full-envelope operands', async (eqlType, spec) => { + const { ops, render } = setup() + const value = sampleFor(spec) + const q = render(await ops.between(matrixColumn(eqlType), value, value)) + + expect(q.sql).toContain( + `eql_v3.gte("matrix_users"."${slug(eqlType)}", $1::jsonb)`, + ) + expect(q.sql).toContain( + `eql_v3.lte("matrix_users"."${slug(eqlType)}", $2::jsonb)`, + ) + expect(q.params).toEqual([TERM_JSON, TERM_JSON]) + }) + + it.each( + orderDomains, + )('%s notBetween wraps the range in NOT (...)', async (eqlType, spec) => { + const { ops, render } = setup() + const value = sampleFor(spec) + const q = render(await ops.notBetween(matrixColumn(eqlType), value, value)) + + expect(q.sql).toMatch(/^not \(/i) + expect(q.sql).toContain( + `eql_v3.gte("matrix_users"."${slug(eqlType)}", $1::jsonb)`, + ) + expect(q.sql).toContain( + `eql_v3.lte("matrix_users"."${slug(eqlType)}", $2::jsonb)`, + ) + expect(q.params).toEqual([TERM_JSON, TERM_JSON]) + }) + + it('not(between(...)) negates the whole range, not just its lower bound', async () => { + const { ops, render } = setup() + + const q = render(ops.not(await ops.between(users.age, 25, 40))) + + // `not eql_v3.gte(..) AND eql_v3.lte(..)` would parse as `(NOT gte) AND + // lte` in Postgres — every row under the lower bound, none of the intended + // complement. The range must arrive pre-parenthesised. + expect(q.sql).toBe( + 'not (eql_v3.gte("users"."age", $1::jsonb) AND eql_v3.lte("users"."age", $2::jsonb))', + ) + }) + + it('between stays parenthesised when combined with other predicates', async () => { + const { ops, render } = setup() + + const q = render( + await ops.or( + ops.between(users.age, 25, 40), + ops.eq(users.nickname, 'ada'), + ), + ) + + expect(q.sql).toContain( + '(eql_v3.gte("users"."age", $1::jsonb) AND eql_v3.lte("users"."age", $2::jsonb))', + ) + }) + + it.each(orderDomains)('%s asc / desc extract the ord term', (eqlType) => { + const { ops, render } = setup() + const ascq = render(ops.asc(matrixColumn(eqlType))) + expect(ascq.sql).toContain( + `eql_v3.ord_term("matrix_users"."${slug(eqlType)}")`, + ) + expect(ascq.sql.toLowerCase()).toContain('asc') + + const descq = render(ops.desc(matrixColumn(eqlType))) + expect(descq.sql).toContain( + `eql_v3.ord_term("matrix_users"."${slug(eqlType)}")`, + ) + expect(descq.sql.toLowerCase()).toContain('desc') + }) +}) + +describe('createEncryptionOperatorsV3 - free-text match', () => { + it.each( + matchDomains, + )('%s contains emits latest eql_v3.contains with a full-envelope operand', async (eqlType, spec) => { + const { ops, encrypt, render } = setup() + const q = render(await ops.contains(matrixColumn(eqlType), sampleFor(spec))) + + expect(q.sql).toContain( + `eql_v3.contains("matrix_users"."${slug(eqlType)}", $1::jsonb)`, + ) + expect(q.params).toEqual([TERM_JSON]) + expect(encrypt.mock.calls[0]?.[1]?.column.getName()).toBe(slug(eqlType)) + }) + + it('negation is expressed through the passthrough Drizzle not operator', async () => { + const { ops, render } = setup() + const q = render(ops.not(await ops.contains(users.email, 'example.com'))) + expect(q.sql).toMatch(/^not /i) + expect(q.sql).toContain('eql_v3.contains("users"."email", $1::jsonb)') + }) + + it('does not expose obsolete like/ilike helpers', () => { + const { ops } = setup() + expect('like' in ops).toBe(false) + expect('ilike' in ops).toBe(false) + expect('notIlike' in ops).toBe(false) + }) +}) + +describe('createEncryptionOperatorsV3 - storage-only domains', () => { + it.each(storageDomains)('%s eq throws', async (eqlType, spec) => { + const { ops } = setup() + await expect( + ops.eq(matrixColumn(eqlType), sampleFor(spec)), + ).rejects.toBeInstanceOf(EncryptionOperatorError) + }) + + it.each(storageDomains)('%s contains throws', async (eqlType, spec) => { + const { ops } = setup() + await expect( + ops.contains(matrixColumn(eqlType), sampleFor(spec)), + ).rejects.toBeInstanceOf(EncryptionOperatorError) + }) + + it.each(storageDomains)('%s asc throws synchronously', (eqlType) => { + const { ops } = setup() + expect(() => ops.asc(matrixColumn(eqlType))).toThrow( + EncryptionOperatorError, + ) + }) +}) + +describe('createEncryptionOperatorsV3 - array, ordering, combinators', () => { + it('inArray ORs one eq term per value; empty array throws', async () => { + const { ops, render } = setup() + const q = render(await ops.inArray(users.nickname, ['ada', 'grace'])) + expect(q.sql).toContain(' or ') + expect((q.sql.match(/eql_v3\.eq/g) ?? []).length).toBe(2) + await expect(ops.inArray(users.nickname, [])).rejects.toThrow(/non-empty/) + }) + + it('inArray fans out more than MAX_IN_ARRAY_CONCURRENCY values exactly once', async () => { + const { ops, encrypt, render } = setup() + const values = [ + 'ada', + 'grace', + 'alan', + 'katherine', + 'dorothy', + 'mary', + 'joan', + ] + + const q = render(await ops.inArray(users.nickname, values)) + + expect((q.sql.match(/eql_v3\.eq/g) ?? []).length).toBe(values.length) + expect(q.params).toEqual(values.map(() => TERM_JSON)) + expect(encrypt).toHaveBeenCalledTimes(values.length) + expect(encrypt.mock.calls.map(([value]) => value).sort()).toEqual( + [...values].sort(), + ) + }) + + it('inArray on an ORE column uses ORE equality for each term', async () => { + const { ops, render } = setup() + const q = render(await ops.inArray(users.age, [30, 42])) + expect(q.sql).toContain(' or ') + expect((q.sql.match(/eql_v3\.eq/g) ?? []).length).toBe(2) + }) + + it('notInArray ANDs one ne term per value; empty array throws', async () => { + const { ops, render } = setup() + const q = render(await ops.notInArray(users.nickname, ['ada', 'grace'])) + expect(q.sql).toContain(' and ') + await expect(ops.notInArray(users.nickname, [])).rejects.toThrow( + /non-empty/, + ) + }) + + it('notInArray on an ORE column uses ORE inequality for each term', async () => { + const { ops, render } = setup() + const q = render(await ops.notInArray(users.age, [30, 42])) + expect(q.sql).toContain(' and ') + expect((q.sql.match(/eql_v3\.neq/g) ?? []).length).toBe(2) + }) + + it('inArray encrypts the whole list in a single bulkEncrypt crossing', async () => { + const { ops, encrypt, bulkEncrypt, render } = setupBulk() + const values = ['ada', 'grace', 'alan', 'katherine', 'dorothy'] + + const q = render(await ops.inArray(users.nickname, values)) + + expect(bulkEncrypt).toHaveBeenCalledTimes(1) + expect(encrypt).not.toHaveBeenCalled() + expect(bulkEncrypt.mock.calls[0]?.[0]).toEqual( + values.map((plaintext) => ({ plaintext })), + ) + const opts = bulkEncrypt.mock.calls[0]?.[1] as { + column: { getName(): string } + } + expect(opts.column.getName()).toBe('nickname') + expect((q.sql.match(/eql_v3\.eq/g) ?? []).length).toBe(values.length) + expect(q.params).toEqual(values.map(() => TERM_JSON)) + }) + + it('notInArray bulk-encrypts once and ANDs one ne term per value', async () => { + const { ops, bulkEncrypt, render } = setupBulk() + + const q = render(await ops.notInArray(users.nickname, ['ada', 'grace'])) + + expect(bulkEncrypt).toHaveBeenCalledTimes(1) + expect((q.sql.match(/eql_v3\.neq/g) ?? []).length).toBe(2) + expect(q.sql).toContain(' and ') + }) + + it('bulk operand encryption carries the lock context and audit config', async () => { + const { ops, bulkEncrypt } = setupBulk() + + await ops.inArray(users.nickname, ['ada', 'grace']) + + const op = bulkEncrypt.mock.results[0]?.value + expect(op.withLockContext).toHaveBeenCalledWith(lockContext) + expect(op.audit).toHaveBeenCalledWith(audit) + }) + + it('bulk terms keep their positions so each eq term matches its value', async () => { + const terms = [{ c: 'ada' }, { c: 'grace' }] as unknown as Array<{ + data: typeof TERM + }> + const { ops, render } = setupBulk(async () => ({ + data: [{ data: terms[0] as never }, { data: terms[1] as never }], + })) + + const q = render(await ops.inArray(users.nickname, ['ada', 'grace'])) + + expect(q.params).toEqual([ + JSON.stringify(terms[0]), + JSON.stringify(terms[1]), + ]) + }) + + it('a bulk encryption failure is wrapped with operator context', async () => { + const { ops } = setupBulk(async () => ({ + failure: { message: 'bad query term' }, + })) + + await expect( + ops.inArray(users.nickname, ['ada', 'grace']), + ).rejects.toMatchObject({ + name: 'EncryptionOperatorError', + context: { columnName: 'nickname', operator: 'inArray' }, + }) + }) + + it('a null value in the list throws before any encryption crossing', async () => { + const { ops, bulkEncrypt } = setupBulk() + + await expect(ops.inArray(users.nickname, ['ada', null])).rejects.toThrow( + /isNull/, + ) + expect(bulkEncrypt).not.toHaveBeenCalled() + }) + + it('a bulk response of the wrong length is rejected rather than silently truncated', async () => { + const { ops } = setupBulk(async () => ({ data: [{ data: TERM }] })) + + // Pin the counts: an off-by-one guard, or a rejection thrown for some + // unrelated reason, must not pass as "handled". + await expect(ops.inArray(users.nickname, ['ada', 'grace'])).rejects.toThrow( + /bulk encryption returned 1 terms for 2 values/, + ) + await expect( + ops.inArray(users.nickname, ['ada', 'grace']), + ).rejects.toBeInstanceOf(EncryptionOperatorError) + }) + + it('inArray gates on the column capability before encrypting anything', async () => { + const { ops, bulkEncrypt } = setupBulk() + + await expect(ops.inArray(users.flag, [true])).rejects.toBeInstanceOf( + EncryptionOperatorError, + ) + expect(bulkEncrypt).not.toHaveBeenCalled() + }) + + it('and ignores undefined conditions and keeps the encrypted predicates', async () => { + const { ops, render } = setup() + const q = render( + await ops.and( + undefined, + ops.eq(users.nickname, 'ada'), + undefined, + ops.gte(users.age, 30), + ), + ) + expect(q.sql).toContain('eql_v3.eq("users"."nickname"') + expect(q.sql).toContain('eql_v3.gte("users"."age"') + expect(q.sql).toContain(' and ') + }) + + it('empty and/or resolve to their neutral predicates', async () => { + const { ops, render } = setup() + expect(render(await ops.and()).sql).toBe('true') + expect(render(await ops.or()).sql).toBe('false') + }) + + it('or combines encrypted and plain predicates', async () => { + const { ops, render } = setup() + const q = render( + await ops.or(ops.eq(users.nickname, 'ada'), drizzleEq(users.id, 1)), + ) + expect(q.sql).toContain(' or ') + expect(q.sql).toContain('eql_v3.eq("users"."nickname"') + expect(q.sql).toContain('"users"."id" = $2') + }) + + it('exports the passthrough Drizzle operators unchanged', () => { + const { ops } = setup() + expect(ops.isNull).toBe(isNull) + expect(ops.isNotNull).toBe(isNotNull) + expect(ops.not).toBe(not) + expect(ops.exists).toBe(exists) + expect(ops.notExists).toBe(notExists) + }) +}) + +describe('createEncryptionOperatorsV3 - gating errors', () => { + it('wraps encryption failures with operator context', async () => { + const { ops } = setup(async () => ({ + failure: { message: 'bad query term' }, + })) + + await expect(ops.eq(users.nickname, 'ada')).rejects.toMatchObject({ + name: 'EncryptionOperatorError', + }) + }) + + it('gt on an equality-only column throws', async () => { + const { ops } = setup() + await expect(ops.gt(users.nickname, 'ada')).rejects.toBeInstanceOf( + EncryptionOperatorError, + ) + }) + + it('contains on a column without match throws', async () => { + const { ops } = setup() + await expect(ops.contains(users.nickname, 'ada')).rejects.toBeInstanceOf( + EncryptionOperatorError, + ) + }) + + it('null operands throw and point callers to null checks', async () => { + const { ops } = setup() + await expect(ops.eq(users.nickname, null)).rejects.toThrow(/isNull/) + await expect(ops.contains(users.email, undefined)).rejects.toThrow(/isNull/) + }) + + it('eq on a storage-only column throws', async () => { + const { ops } = setup() + await expect(ops.eq(users.flag, true)).rejects.toBeInstanceOf( + EncryptionOperatorError, + ) + }) + + it('the equality gate reports the offending domain, as every other gate does', async () => { + const { ops } = setup() + + // Same diagnostic shape as the ore/match gates: operator, capability, + // column, and the domain that cannot answer it. + await expect(ops.eq(users.flag, true)).rejects.toThrow( + 'Operator "eq" requires equality on column "flag" (domain public.boolean does not support it).', + ) + await expect(ops.gt(users.nickname, 'ada')).rejects.toThrow( + 'Operator "gt" requires order/range on column "nickname" (domain public.text_eq does not support it).', + ) + }) + + it('asc on a non-ore column throws synchronously', () => { + const { ops } = setup() + expect(() => ops.asc(users.nickname)).toThrow(EncryptionOperatorError) + }) + + it('a non-v3 column throws', async () => { + const { ops } = setup() + await expect(ops.eq(users.id, 1)).rejects.toBeInstanceOf( + EncryptionOperatorError, + ) + }) + + it('does not treat SQLWrapper objects with column-shaped fields as Drizzle columns', async () => { + const { ops } = setup() + const spoof = { + name: 'nickname', + table: users, + getSQL: () => sql`"users"."nickname"`, + } as unknown as SQLWrapper + + await expect(ops.eq(spoof, 'ada')).rejects.toMatchObject({ + name: 'EncryptionOperatorError', + context: { columnName: 'unknown', operator: 'eq' }, + }) + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/schema-extraction.test.ts b/packages/stack/__tests__/drizzle-v3/schema-extraction.test.ts new file mode 100644 index 000000000..2b7927593 --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/schema-extraction.test.ts @@ -0,0 +1,111 @@ +import { integer, pgTable } from 'drizzle-orm/pg-core' +import { describe, expect, it } from 'vitest' +import { encryptedTable, types as v3Types } from '@/eql/v3' +import { extractEncryptionSchemaV3 } from '@/eql/v3/drizzle/schema-extraction' +import { types } from '@/eql/v3/drizzle/types' + +describe('extractEncryptionSchemaV3', () => { + it('rebuilds an equivalent eql/v3 encryptedTable from every drizzle v3 factory', () => { + const drizzleColumns = Object.fromEntries( + Object.entries(types).map(([factoryName, factory]) => [ + factoryName, + factory(factoryName), + ]), + ) + const authoredColumns = Object.fromEntries( + Object.entries(v3Types).map(([factoryName, factory]) => [ + factoryName, + factory(factoryName), + ]), + ) + + const extracted = extractEncryptionSchemaV3( + pgTable('users', drizzleColumns as never), + ) + const authored = encryptedTable('users', authoredColumns as never) + + expect(extracted.build()).toEqual(authored.build()) + }) + + it('rebuilds an equivalent eql/v3 encryptedTable from a drizzle table', () => { + const users = pgTable('users', { + id: integer().primaryKey(), + email: types.TextSearch('email'), + age: types.IntegerOrd('age'), + }) + + const extracted = extractEncryptionSchemaV3(users) + const authored = encryptedTable('users', { + email: v3Types.TextSearch('email'), + age: v3Types.IntegerOrd('age'), + }) + + expect(extracted.build()).toEqual(authored.build()) + }) + + it('keeps same-named columns on different tables bound to their own v3 domains', () => { + const accounts = pgTable('accounts', { + email: types.TextEq('email'), + }) + const metrics = pgTable('metrics', { + email: types.IntegerOrd('email'), + }) + + const accountsSchema = extractEncryptionSchemaV3(accounts) + const metricsSchema = extractEncryptionSchemaV3(metrics) + + expect(accountsSchema.build()).toEqual( + encryptedTable('accounts', { + email: v3Types.TextEq('email'), + }).build(), + ) + expect(metricsSchema.build()).toEqual( + encryptedTable('metrics', { + email: v3Types.IntegerOrd('email'), + }).build(), + ) + }) + + it('uses the JS property key while preserving distinct SQL column names', () => { + const users = pgTable('users', { + createdOn: types.Date('created_on'), + emailAddress: types.TextEq('email_address'), + }) + + expect(extractEncryptionSchemaV3(users).build()).toEqual( + encryptedTable('users', { + createdOn: v3Types.Date('created_on'), + emailAddress: v3Types.TextEq('email_address'), + }).build(), + ) + }) + + it('uses the Drizzle column name when rebuilding fallback SQL-type columns', () => { + const table = { + [Symbol.for('drizzle:Name')]: 'users', + createdOn: { + name: 'created_on', + getSQLType: () => 'public.date', + }, + } + + expect(extractEncryptionSchemaV3(table as never).build()).toEqual( + encryptedTable('users', { + createdOn: v3Types.Date('created_on'), + }).build(), + ) + }) + + it('throws when the table has no encrypted v3 columns', () => { + const plain = pgTable('plain', { id: integer() }) + expect(() => extractEncryptionSchemaV3(plain)).toThrow(/no encrypted v3/i) + }) + + it('throws the table-name error before checking for encrypted columns', () => { + expect(() => + extractEncryptionSchemaV3({ + secret: { getSQLType: () => 'public.text_eq', name: 'secret' }, + } as never), + ).toThrow('Unable to read table name from Drizzle table.') + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/sql-dialect.test.ts b/packages/stack/__tests__/drizzle-v3/sql-dialect.test.ts new file mode 100644 index 000000000..ba139daf0 --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/sql-dialect.test.ts @@ -0,0 +1,85 @@ +import { not, sql } from 'drizzle-orm' +import { PgDialect } from 'drizzle-orm/pg-core' +import { describe, expect, it } from 'vitest' +import { EQL_V3_FN_SCHEMA, v3Dialect } from '@/eql/v3/drizzle/sql-dialect' + +const dialect = new PgDialect() +const render = (s: ReturnType) => dialect.sqlToQuery(s).sql +const col = sql`"users"."x"` +const enc = sql`${'{"v":"t"}'}` + +describe('v3Dialect', () => { + it('equality via HMAC', () => { + expect(render(v3Dialect.equality('eq', col, enc))).toBe( + 'eql_v3.eq("users"."x", $1::jsonb)', + ) + }) + + it('inequality via HMAC', () => { + expect(render(v3Dialect.equality('ne', col, enc))).toBe( + 'eql_v3.neq("users"."x", $1::jsonb)', + ) + }) + + it('comparison via ORE', () => { + expect(render(v3Dialect.comparison('gte', col, enc))).toBe( + 'eql_v3.gte("users"."x", $1::jsonb)', + ) + }) + + it('range via ORE, parenthesised so it survives negation', () => { + const lo = sql`${'{"v":"lo"}'}` + const hi = sql`${'{"v":"hi"}'}` + expect(render(v3Dialect.range(col, lo, hi))).toBe( + '(eql_v3.gte("users"."x", $1::jsonb) AND eql_v3.lte("users"."x", $2::jsonb))', + ) + }) + + it('a negated range negates the whole conjunction, not just its first term', () => { + const lo = sql`${'{"v":"lo"}'}` + const hi = sql`${'{"v":"hi"}'}` + + // Drizzle's `not` renders `not ${condition}` with no parentheses of its + // own. Postgres binds NOT tighter than AND, so an unparenthesised range + // would silently become `(NOT gte) AND lte`. + expect(render(not(v3Dialect.range(col, lo, hi)))).toBe( + 'not (eql_v3.gte("users"."x", $1::jsonb) AND eql_v3.lte("users"."x", $2::jsonb))', + ) + }) + + it('contains via two-arg function', () => { + expect(render(v3Dialect.contains(col, enc))).toBe( + 'eql_v3.contains("users"."x", $1::jsonb)', + ) + }) + + it('orderBy extracts the ord term', () => { + expect(render(v3Dialect.orderBy(col))).toBe('eql_v3.ord_term("users"."x")') + }) + + it('every helper schema-qualifies its function call', () => { + const lo = sql`${'{"v":"lo"}'}` + const hi = sql`${'{"v":"hi"}'}` + const fragments = [ + render(v3Dialect.equality('eq', col, enc)), + render(v3Dialect.equality('ne', col, enc)), + render(v3Dialect.comparison('gte', col, enc)), + render(v3Dialect.contains(col, enc)), + render(v3Dialect.orderBy(col)), + ] + + // Deliberately asserts the LITERAL prefix, not `${EQL_V3_FN_SCHEMA}.`. + // Deriving the expectation from the constant under test would hold for any + // value of that constant — including a wrong one. + for (const fragment of fragments) { + expect(fragment.startsWith('eql_v3.')).toBe(true) + } + // `range` invokes two functions, neither of them leading the fragment. + expect( + (render(v3Dialect.range(col, lo, hi)).match(/eql_v3\./g) ?? []).length, + ).toBe(2) + // The constant is the single knob those literals must agree with, so a + // one-line schema move fails here first and loudly. + expect(EQL_V3_FN_SCHEMA).toBe('eql_v3') + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/types.test-d.ts b/packages/stack/__tests__/drizzle-v3/types.test-d.ts new file mode 100644 index 000000000..e33b4f7a0 --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/types.test-d.ts @@ -0,0 +1,42 @@ +import { describe, expectTypeOf, it } from 'vitest' +import type { types as v3Types } from '@/eql/v3' +import { types } from '@/eql/v3/drizzle/types' +import type { Encrypted } from '@/types' + +describe('v3 drizzle types - type-level', () => { + it('exposes exactly the same factory keys as @/eql/v3 types', () => { + expectTypeOf().toEqualTypeOf() + }) + + it('types the data slot as the encrypted envelope, not plaintext (A3)', () => { + // The value stored/inserted/selected is the ENCRYPTED EQL v3 jsonb envelope, + // NOT the column's plaintext. So every concrete domain — regardless of its + // plaintext axis (number/Date/boolean/string) — exposes `Encrypted` in its + // Drizzle `data` slot. Plaintext inference is a separate concern, proven on + // the v3 core builder via `PlaintextForColumn` (see schema-v3.test-d.ts). + expectTypeOf(types.IntegerOrd('age')._.data).toEqualTypeOf() + expectTypeOf(types.IntegerEq('age_eq')._.data).toEqualTypeOf() + expectTypeOf(types.BigintOrd('big')._.data).toEqualTypeOf() + expectTypeOf(types.DateOrd('created_at')._.data).toEqualTypeOf() + expectTypeOf(types.Timestamp('ts')._.data).toEqualTypeOf() + expectTypeOf(types.Boolean('flag')._.data).toEqualTypeOf() + expectTypeOf(types.TextEq('nickname')._.data).toEqualTypeOf() + expectTypeOf(types.TextSearch('search')._.data).toEqualTypeOf() + expectTypeOf(types.DoubleOrd('d')._.data).toEqualTypeOf() + }) + + it('does not expose obsolete pre-0.27 concrete domain names', () => { + // @ts-expect-error - use IntegerOrd + types.Int4Ord('age') + // @ts-expect-error - use SmallintOrd + types.Int2Ord('age') + // @ts-expect-error - use Timestamp + types.Timestamptz('created_at') + // @ts-expect-error - use Boolean + types.Bool('active') + // @ts-expect-error - use RealEq + types.Float4Eq('score') + // @ts-expect-error - use DoubleOrd + types.Float8Ord('score') + }) +}) diff --git a/packages/stack/__tests__/drizzle-v3/types.test.ts b/packages/stack/__tests__/drizzle-v3/types.test.ts new file mode 100644 index 000000000..c143fc17c --- /dev/null +++ b/packages/stack/__tests__/drizzle-v3/types.test.ts @@ -0,0 +1,22 @@ +import { describe, expect, it } from 'vitest' +import { types as v3Types } from '@/eql/v3' +import { getEqlV3Column } from '@/eql/v3/drizzle/column' +import { types } from '@/eql/v3/drizzle/types' + +describe('v3 drizzle types namespace', () => { + it('exposes the same factory names as @/eql/v3 types', () => { + expect(Object.keys(types).sort()).toEqual(Object.keys(v3Types).sort()) + }) + + it.each( + Object.entries(types), + )('%s mirrors the authoring DSL and recovers the concrete eql type', (factoryName, factory) => { + const drizzleColumn = factory(factoryName) + const authoredColumn = + v3Types[factoryName as keyof typeof v3Types](factoryName) + + expect(getEqlV3Column(factoryName, drizzleColumn)?.getEqlType()).toBe( + authoredColumn.getEqlType(), + ) + }) +}) diff --git a/packages/stack/__tests__/helpers/live-gate.ts b/packages/stack/__tests__/helpers/live-gate.ts index 8ad3675ec..a674fbebb 100644 --- a/packages/stack/__tests__/helpers/live-gate.ts +++ b/packages/stack/__tests__/helpers/live-gate.ts @@ -23,6 +23,17 @@ export const LIVE_EQL_V3_PG_ENABLED = Boolean( process.env.DATABASE_URL && LIVE_CIPHERSTASH_ENABLED, ) +/** + * True when live credentials AND a `USER_JWT` are configured. The identity / + * lock-context live suites additionally require a `USER_JWT` to bind keys to an + * end-user identity, and SOFT-SKIP (inline `if (!userJwt) return`) when it is + * absent — so a missing/rotated `USER_JWT` lets them skip green in CI. This + * flag lets the live-coverage guard assert that path is actually exercised. + */ +export const LIVE_LOCK_CONTEXT_ENABLED = Boolean( + process.env.USER_JWT && LIVE_CIPHERSTASH_ENABLED, +) + export const describeLive = LIVE_CIPHERSTASH_ENABLED ? describe : describe.skip export const describeLivePg = LIVE_EQL_V3_PG_ENABLED ? describe : describe.skip diff --git a/packages/stack/__tests__/live-coverage-guard.test.ts b/packages/stack/__tests__/live-coverage-guard.test.ts index aed0ad6de..6b6005ba6 100644 --- a/packages/stack/__tests__/live-coverage-guard.test.ts +++ b/packages/stack/__tests__/live-coverage-guard.test.ts @@ -55,6 +55,7 @@ import { describe, expect, it } from 'vitest' import { LIVE_CIPHERSTASH_ENABLED, LIVE_EQL_V3_PG_ENABLED, + LIVE_LOCK_CONTEXT_ENABLED, } from './helpers/live-gate' // GitHub Actions always sets CI=true; treat any truthy CI as "must run live". @@ -87,6 +88,28 @@ describe('live-coverage guard', () => { }, ) + // DEFERRED (follow-up): the CI `USER_JWT` secret is not yet provisioned, so + // enforcing this now would fail CI. Skipped until the secret exists — flip + // back to `it.runIf(IN_CI)` at that point. It still documents the real gap: + // the identity / lock-context live suites soft-skip on a missing USER_JWT, so + // once the secret lands this guard turns a silent whole-suite skip into a + // loud failure (as the CS_*/DATABASE_URL guards already do). + it.skip( + 'CI must have USER_JWT so the lock-context live suites do not silently skip', + () => { + expect( + LIVE_LOCK_CONTEXT_ENABLED, + 'CI must run the live lock-context / identity suites — ' + + '`LIVE_LOCK_CONTEXT_ENABLED` is false. This needs the CS_* creds AND ' + + 'a `USER_JWT`; the identity/lock-context suites (e.g. ' + + 'lock-context.test.ts, protect-ops.test.ts, ' + + 'operators-lock-context-live-pg.test.ts) SOFT-SKIP when USER_JWT is ' + + 'absent, so a missing/rotated USER_JWT lets them skip green with no ' + + 'signal — the exact failure mode this guard exists to prevent.', + ).toBe(true) + }, + ) + // Local dev with no creds: nothing to assert. Keep at least one always-run // assertion so the file is never reported as fully empty/pending. it('is always collected (guard file runs outside every live gate)', () => { diff --git a/packages/stack/__tests__/schema-v3.test.ts b/packages/stack/__tests__/schema-v3.test.ts index 18d9fa619..c5620407e 100644 --- a/packages/stack/__tests__/schema-v3.test.ts +++ b/packages/stack/__tests__/schema-v3.test.ts @@ -22,65 +22,19 @@ describe('eql_v3 text_search column', () => { expect(v3).toStrictEqual(v2) }) - it('.freeTextSearch(opts) overrides each provided key and keeps the rest as defaults', () => { - const built = types - .TextSearch('email') - .freeTextSearch({ - tokenizer: { kind: 'ngram', token_length: 4 }, - k: 8, - m: 4096, - include_original: false, - }) - .build() + it('default build() emits the unique, ore, and match indexes', () => { + const built = types.TextSearch('email').build() + expect(built.indexes.unique).toEqual({ token_filters: [] }) + expect(built.indexes.ore).toEqual({}) expect(built.indexes.match).toEqual({ - tokenizer: { kind: 'ngram', token_length: 4 }, - // omitted -> default downcase filter retained + tokenizer: { kind: 'ngram', token_length: 3 }, token_filters: [{ kind: 'downcase' }], - k: 8, - m: 4096, - include_original: false, + k: 6, + m: 2048, + include_original: true, }) }) - it('.freeTextSearch({ token_filters: [] }) overrides the downcase default with an empty array', () => { - // LOAD-BEARING: `[] ?? default` evaluates to `[]` (an empty array is not - // nullish), so an explicit empty array must OVERRIDE the downcase default, - // not fall back to it. Mirrors v2 (schema-builders.test.ts). - const built = types - .TextSearch('email') - .freeTextSearch({ token_filters: [] }) - .build() - expect(built.indexes.match.token_filters).toEqual([]) - }) - - it('repeated .freeTextSearch() calls are last-call-wins-fully (each re-merges against defaults, not prior state)', () => { - // Each call re-merges against a fresh defaultMatchOpts(), not the - // accumulated matchOpts — so the second call resets k back to its default - // of 6. This is intentional: it mirrors v2 exactly. Pinned here so a future - // "merge against current state" change can't silently slip in. - const built = types - .TextSearch('email') - .freeTextSearch({ k: 8 }) - .freeTextSearch({ m: 4096 }) - .build() - expect(built.indexes.match.k).toBe(6) - expect(built.indexes.match.m).toBe(4096) - }) - - it('.freeTextSearch() with no argument is a no-op: build() equals the default build()', () => { - // Pins the opts === undefined branch: every `opts?.x ?? default` falls - // through, so a bare call must emit exactly the default match block. - expect(types.TextSearch('email').freeTextSearch().build()).toStrictEqual( - types.TextSearch('email').build(), - ) - }) - - it('.freeTextSearch() is tuning-only: unique and ore indexes stay present', () => { - const built = types.TextSearch('email').freeTextSearch({ k: 8 }).build() - expect(built.indexes.unique).toEqual({ token_filters: [] }) - expect(built.indexes.ore).toEqual({}) - }) - it('built columns share no mutable state: mutating one build() output does not affect another', () => { // Guards against the shared-defaults aliasing bug: defaults come from a // per-instance factory and build() deep-clones the match block. @@ -104,29 +58,6 @@ describe('eql_v3 text_search column', () => { expect(c.indexes.match.k).toBe(6) expect(c.indexes.match.token_filters).toEqual([{ kind: 'downcase' }]) }) - - it('clones caller opts on freeTextSearch(): mutating them before build() does not leak', () => { - // build() deep-clones at build time, but if freeTextSearch stored the - // caller's nested tokenizer / token_filters by reference, a caller mutating - // their own opts object between freeTextSearch(opts) and build() would leak - // the mutation into the emitted config. freeTextSearch must clone on write. - const opts = { - tokenizer: { kind: 'ngram' as const, token_length: 3 }, - token_filters: [{ kind: 'downcase' as const }], - } - const col = types.TextSearch('email').freeTextSearch(opts) - - // Mutate the caller's own opts AFTER freeTextSearch but BEFORE build(). - opts.tokenizer.token_length = 999 - opts.token_filters.push({ kind: 'downcase' as const }) - - const built = col.build() - expect(built.indexes.match.tokenizer).toEqual({ - kind: 'ngram', - token_length: 3, - }) - expect(built.indexes.match.token_filters).toEqual([{ kind: 'downcase' }]) - }) }) describe('eql_v3 text_match column', () => { diff --git a/packages/stack/__tests__/typed-client-v3.test.ts b/packages/stack/__tests__/typed-client-v3.test.ts index c605cdec2..5969989b7 100644 --- a/packages/stack/__tests__/typed-client-v3.test.ts +++ b/packages/stack/__tests__/typed-client-v3.test.ts @@ -86,4 +86,40 @@ describe('typedClient — decrypt reconstruction', () => { const result = await client.decryptModel({}, table) expect(result.failure).toBeTruthy() }) + + it('fails with a DecryptionError when given a table it was not initialized with', async () => { + const other = encryptedTable('other', { x: types.Text('x') }) + const client = typedClient(fakeClient({ when: null }), table) + + const single = await client.decryptModel({}, other as never) + expect(single.failure?.type).toBe('DecryptionError') + expect(single.failure?.message).toMatch(/not initialized with/i) + + const bulk = await client.bulkDecryptModels([{}], other as never) + expect(bulk.failure?.type).toBe('DecryptionError') + }) + + // Reconstructors are keyed by `tableName`, not object identity: a table + // re-imported from another module (or rebuilt across an HMR reload) is a + // distinct object that still satisfies `Table extends S[number]`. + it('decrypts when handed a structurally identical, separately constructed table', async () => { + const sameTableRebuilt = encryptedTable('t', { + when: types.Timestamp('when'), + note: types.Text('note'), + createdOn: types.Date('created_on'), + }) + expect(sameTableRebuilt).not.toBe(table) + + const client = typedClient( + fakeClient({ when: '2020-01-02T03:04:05.000Z', note: 'hi' }), + table, + ) + + const result = await client.decryptModel({}, sameTableRebuilt) + expect(result.failure).toBeFalsy() + if (result.failure) return + + const data = result.data as Record + expect(data.when).toBeInstanceOf(Date) + }) }) diff --git a/packages/stack/__tests__/v3-matrix/catalog.ts b/packages/stack/__tests__/v3-matrix/catalog.ts index c2dbf8034..7dbe95046 100644 --- a/packages/stack/__tests__/v3-matrix/catalog.ts +++ b/packages/stack/__tests__/v3-matrix/catalog.ts @@ -123,6 +123,20 @@ export function typedEntries( return Object.entries(obj) as Array<[K, V]> } +/** The schema the concrete EQL v3 domains live in — see `EqlV3TypeName`. */ +export const EQL_V3_DOMAIN_SCHEMA = 'public' + +/** + * A `public.integer_ord` domain reduced to a bare `integer_ord`, suitable as a + * column identifier in the test tables. Shared by every suite that builds a + * table from the matrix, so a future domain-schema move is a one-line change + * rather than five. + */ +export function eqlTypeSlug(eqlType: EqlV3TypeName | string): string { + const prefix = `${EQL_V3_DOMAIN_SCHEMA}.` + return eqlType.startsWith(prefix) ? eqlType.slice(prefix.length) : eqlType +} + // Capability shorthands (mirror the SDK's internal presets). const STORAGE = { equality: false, diff --git a/packages/stack/__tests__/v3-matrix/matrix-live-pg.test.ts b/packages/stack/__tests__/v3-matrix/matrix-live-pg.test.ts index 1c587cdfc..1252b8378 100644 --- a/packages/stack/__tests__/v3-matrix/matrix-live-pg.test.ts +++ b/packages/stack/__tests__/v3-matrix/matrix-live-pg.test.ts @@ -63,6 +63,7 @@ import { describeLivePg, LIVE_EQL_V3_PG_ENABLED } from '../helpers/live-gate' import { type DomainSpec, type EqlV3TypeName, + eqlTypeSlug as slug, typedEntries, V3_MATRIX, } from './catalog' @@ -84,12 +85,6 @@ const sql = LIVE_EQL_V3_PG_ENABLED const TABLE_NAME = 'v3_matrix_live_pg' const TEST_RUN_ID = `matrix-live-pg-${Date.now()}-${Math.random().toString(36).slice(2, 8)}` -/** `public.integer_ord` -> `integer_ord`: a valid, unique Postgres column name. - * The domains were renamed `eql_v3.* -> public.*`, so strip the `public.` - * prefix — a leftover dot in the column name breaks both the FFI's identifier - * resolution and the `"col" ` DDL below. */ -const slug = (t: EqlV3TypeName): string => t.replace(/^public\./, '') - const expectDecryptedStorageValue = ( decrypted: unknown, expected: unknown, @@ -308,10 +303,13 @@ beforeAll(async () => { // are built dynamically (`Object.fromEntries`) and carry no static type. const encryptOperand = async (t: EqlV3TypeName, value: unknown) => unwrapResult( - await client.encrypt(value as never, { - table, - column: columnRef(t), - } as never), + await client.encrypt( + value as never, + { + table, + column: columnRef(t), + } as never, + ), ) for (const [t, spec] of eqDomains) { eqTerms[slug(t)] = await encryptOperand(t, spec.samples[0]) @@ -475,7 +473,9 @@ describeLivePg('v3 matrix live Postgres coverage (all 35 domains)', () => { .sort((x, y) => comparePlaintext(x.value, y.value)) .map((entry) => entry.id) - const pairs = await sql.unsafe>( + const pairs = await sql.unsafe< + Array<{ a: number; b: number; lt: boolean }> + >( `SELECT x.id AS a, y.id AS b, eql_v3.lt(x."${col}", y."${col}") AS lt FROM ${TABLE_NAME} x CROSS JOIN ${TABLE_NAME} y diff --git a/packages/stack/__tests__/v3-matrix/matrix-live.test.ts b/packages/stack/__tests__/v3-matrix/matrix-live.test.ts index 5e66867bd..c3189cace 100644 --- a/packages/stack/__tests__/v3-matrix/matrix-live.test.ts +++ b/packages/stack/__tests__/v3-matrix/matrix-live.test.ts @@ -25,16 +25,11 @@ import { describeLive } from '../helpers/live-gate' import { type DomainSpec, type EqlV3TypeName, + eqlTypeSlug as slug, typedEntries, V3_MATRIX, } from './catalog' -/** `public.integer_ord` -> `integer_ord`: a valid, per-domain-unique column name. - * The domains were renamed `eql_v3.* -> public.*`, so strip the `public.` - * prefix — a leftover dot in the column name (`public.integer_ord`) makes the - * FFI's identifier resolution fail every encrypt. */ -const slug = (t: EqlV3TypeName): string => t.replace(/^public\./, '') - // `as const satisfies Record<...>` gives `V3_MATRIX` a narrower type than // `Record` (rows that omit the optional // `errorSamples` field literally lack that key, rather than typing it diff --git a/packages/stack/package.json b/packages/stack/package.json index 600bf5758..28ccc105f 100644 --- a/packages/stack/package.json +++ b/packages/stack/package.json @@ -54,6 +54,9 @@ "eql/v3": [ "./dist/eql/v3/index.d.ts" ], + "eql/v3/drizzle": [ + "./dist/eql/v3/drizzle/index.d.ts" + ], "v3": [ "./dist/encryption/v3.d.ts" ], @@ -135,6 +138,16 @@ "default": "./dist/eql/v3/index.cjs" } }, + "./eql/v3/drizzle": { + "import": { + "types": "./dist/eql/v3/drizzle/index.d.ts", + "default": "./dist/eql/v3/drizzle/index.js" + }, + "require": { + "types": "./dist/eql/v3/drizzle/index.d.cts", + "default": "./dist/eql/v3/drizzle/index.cjs" + } + }, "./v3": { "import": { "types": "./dist/encryption/v3.d.ts", diff --git a/packages/stack/src/encryption/operations/bulk-encrypt.ts b/packages/stack/src/encryption/operations/bulk-encrypt.ts index 6e3ec802d..3970fea0e 100644 --- a/packages/stack/src/encryption/operations/bulk-encrypt.ts +++ b/packages/stack/src/encryption/operations/bulk-encrypt.ts @@ -1,6 +1,7 @@ import { type Result, withResult } from '@byteslice/result' import { encryptBulk, type JsPlaintext } from '@cipherstash/protect-ffi' import { getErrorCode } from '@/encryption/helpers/error-code' +import { assertValidNumericValue } from '@/encryption/helpers/validation' import { type EncryptionError, EncryptionErrorTypes } from '@/errors' import { type Context, @@ -23,6 +24,12 @@ import { EncryptionOperation } from './base-operation' // Drops nulls so they don't reach protect-ffi (which would otherwise // produce a SteVec wrapping the JSON null). The dropped positions are // re-inserted as null in `mapEncryptedDataToResult`. +// +// Each surviving plaintext is validated exactly as `EncryptOperation` validates +// its single operand: NaN / ±Infinity / out-of-int64 `bigint` are rejected +// client-side, because protect-ffi's behaviour on such a value is unobservable. +// Callers that batch instead of looping (the v3 Drizzle `inArray`, for one) +// must not lose that guard by choosing the bulk path. const createEncryptPayloads = ( plaintexts: BulkEncryptPayload, column: BuildableColumn, @@ -31,13 +38,16 @@ const createEncryptPayloads = ( ) => { return plaintexts .filter(({ plaintext }) => plaintext !== null) - .map(({ id, plaintext }) => ({ - id, - plaintext: plaintext as JsPlaintext, - column: column.getName(), - table: table.tableName, - ...(lockContext && { lockContext }), - })) + .map(({ id, plaintext }) => { + assertValidNumericValue(plaintext) + return { + id, + plaintext: plaintext as JsPlaintext, + column: column.getName(), + table: table.tableName, + ...(lockContext && { lockContext }), + } + }) } const createNullResult = (plaintexts: BulkEncryptPayload): BulkEncryptedData => diff --git a/packages/stack/src/encryption/v3.ts b/packages/stack/src/encryption/v3.ts index 29be610aa..acf4b1b82 100644 --- a/packages/stack/src/encryption/v3.ts +++ b/packages/stack/src/encryption/v3.ts @@ -180,23 +180,27 @@ export function typedClient( // `bulkDecryptModels` therefore never call `build()` (whose throw would surface // as a promise rejection and break their `Promise>` contract) and no // longer rebuild the row-invariant config on every call. + // Keyed by `tableName`, not table object identity: `AnyV3Table` is + // structurally typed, so a table re-imported from another module (or rebuilt + // after an HMR reload) satisfies `Table extends S[number]` yet is a different + // object. Identity keying would fail those valid calls. `tableName` is the + // semantic identity the FFI encrypt config and `build()` already key on. const reconstructors = new Map< - AnyV3Table, + string, (row: Record) => Record >() for (const table of schemas) { - reconstructors.set(table, rowReconstructor(table)) + reconstructors.set(table.tableName, rowReconstructor(table)) } - // A table not among the schemas this client was initialized with (only - // reachable by bypassing the `Table extends S[number]` type constraint) has no + // A table not among the schemas this client was initialized with has no // precomputed reconstructor. Return a Result failure rather than building one // inline, which could throw and reject the Result-shaped decrypt promise. const unknownTableFailure: { failure: EncryptionError } = { failure: { type: EncryptionErrorTypes.DecryptionError, message: - '[eql/v3]: decryptModel received a table this client was not initialized with — pass the same table object(s) given to EncryptionV3/typedClient', + '[eql/v3]: decryptModel received a table this client was not initialized with — pass a table given to EncryptionV3/typedClient', }, } @@ -211,7 +215,7 @@ export function typedClient( client.bulkEncryptModels(input as never, table as never) as never, decrypt: (encrypted) => client.decrypt(encrypted), decryptModel: async (input, table, lockContext) => { - const reconstruct = reconstructors.get(table) + const reconstruct = reconstructors.get(table.tableName) if (!reconstruct) return unknownTableFailure as never const op = client.decryptModel(input as never) const result = await (lockContext ? op.withLockContext(lockContext) : op) @@ -219,7 +223,7 @@ export function typedClient( return { data: reconstruct(result.data) } as never }, bulkDecryptModels: async (input, table, lockContext) => { - const reconstruct = reconstructors.get(table) + const reconstruct = reconstructors.get(table.tableName) if (!reconstruct) return unknownTableFailure as never const op = client.bulkDecryptModels(input as never) const result = await (lockContext ? op.withLockContext(lockContext) : op) diff --git a/packages/stack/src/eql/v3/columns.ts b/packages/stack/src/eql/v3/columns.ts index 2753ef923..a48fa0b33 100644 --- a/packages/stack/src/eql/v3/columns.ts +++ b/packages/stack/src/eql/v3/columns.ts @@ -1,10 +1,5 @@ -import type { ColumnSchema, MatchIndexOpts } from '@/schema' -import { - type BuiltMatchIndexOpts, - cloneMatchOpts, - defaultMatchOpts, - resolveMatchOpts, -} from '@/schema/match-defaults' +import type { ColumnSchema } from '@/schema' +import { defaultMatchOpts } from '@/schema/match-defaults' /** * The query capabilities a v3 concrete domain exposes. These are SDK-facing @@ -425,8 +420,8 @@ const TEXT_SEARCH_DOMAIN = { * Builder for a `public.text_search` column. * * The concrete type inherently enables equality + order/range + free-text - * match — there are no capability-enabling methods. `.freeTextSearch(opts?)` - * tunes the match index only. + * match — there are no capability-enabling or tuning methods. The match index + * is always emitted with the default configuration. * * NOTE — querying: a `text_search` column emits all three indexes (`unique`, * `ore`, `match`), and the shared index-inference picks them by fixed priority @@ -445,45 +440,8 @@ const TEXT_SEARCH_DOMAIN = { export class EncryptedTextSearchColumn extends EncryptedV3Column< typeof TEXT_SEARCH_DOMAIN > { - private matchOpts: BuiltMatchIndexOpts - constructor(columnName: string) { super(columnName, TEXT_SEARCH_DOMAIN) - this.matchOpts = defaultMatchOpts() - } - - /** - * Tune the match index. Each provided key replaces its default; omitted - * keys keep the default. This NEVER enables a capability — match is always - * on for this type. Merge semantics mirror v2's `opts?.x ?? default`. - */ - freeTextSearch(opts?: MatchIndexOpts): this { - // Shared merge+clone (schema/match-defaults) — one source of truth with the - // v2 `freeTextSearch()` builder. `resolveMatchOpts` merges each key over the - // per-call defaults and deep-clones, so a caller mutating their own opts - // object between `freeTextSearch(opts)` and `build()` cannot leak into the - // emitted config (clone-on-write). - this.matchOpts = resolveMatchOpts(opts) - return this - } - - /** Emit the encrypt-config column. Byte-identical to a v2 equality+order+match column. */ - override build(): ColumnSchema { - // Derive `cast_as` + the `unique`/`ore` blocks from the shared - // capability→index mapping (this domain's TEXT_SEARCH capabilities produce - // exactly `{ unique, ore, match }`), then override ONLY `match` with this - // builder's tuned opts. Hand-writing the unique/ore shape here would let it - // silently drift from `indexesForCapabilities` — the exact divergence class - // behind the text_ord `hm`-index bug. Deep-clone the match block so the - // returned config never aliases this builder's internal `matchOpts`. - const base = super.build() - return { - ...base, - indexes: { - ...base.indexes, - match: cloneMatchOpts(this.matchOpts), - }, - } } } diff --git a/packages/stack/src/eql/v3/drizzle/codec.ts b/packages/stack/src/eql/v3/drizzle/codec.ts new file mode 100644 index 000000000..43973ef03 --- /dev/null +++ b/packages/stack/src/eql/v3/drizzle/codec.ts @@ -0,0 +1,44 @@ +/** + * Codec for the value actually stored in a v3 column: the ENCRYPTED EQL v3 + * envelope ({@link Encrypted}), not plaintext. v3 columns are + * `CREATE DOMAIN ... AS jsonb`, so the envelope serialises as plain jsonb — + * distinct from v2's composite-literal parser. + */ + +import type { Encrypted } from '@/types' + +/** + * `JSON.stringify` replacer that renders any stray `bigint` as its decimal + * string instead of throwing `TypeError: Do not know how to serialize a + * BigInt`. A well-formed EQL v3 envelope never carries a `bigint` — `bigint` + * plaintext is encrypted to ciphertext strings before it ever reaches this + * codec — so this is a defensive guard against a malformed envelope, never a + * data path. + */ +function bigintSafeReplacer(_key: string, value: unknown): unknown { + return typeof value === 'bigint' ? value.toString() : value +} + +/** Serialise an encrypted envelope to a jsonb string for the driver. Null and + * undefined map to SQL NULL (JS `null`), never the JSON `null` literal. */ +export function v3ToDriver(value: Encrypted | null | undefined): string | null { + if (value === null || value === undefined) { + return null + } + return JSON.stringify(value, bigintSafeReplacer) +} + +/** Parse a driver jsonb value back into an encrypted envelope. `postgres` + * hands back an already-parsed object for jsonb; a string is parsed. Null and + * undefined normalise to `null` (SQL NULL). */ +export function v3FromDriver( + value: string | object | null | undefined, +): Encrypted | null { + if (value === null || value === undefined) { + return null + } + if (typeof value === 'object') { + return value as Encrypted + } + return JSON.parse(value) as Encrypted +} diff --git a/packages/stack/src/eql/v3/drizzle/column.ts b/packages/stack/src/eql/v3/drizzle/column.ts new file mode 100644 index 000000000..abc2be4c2 --- /dev/null +++ b/packages/stack/src/eql/v3/drizzle/column.ts @@ -0,0 +1,121 @@ +import { customType } from 'drizzle-orm/pg-core' +import { type AnyEncryptedV3Column, types as v3Types } from '@/eql/v3' +import type { Encrypted } from '@/types' +import { v3FromDriver, v3ToDriver } from './codec.js' + +const buildersByDomain: ReadonlyMap< + string, + (name: string) => AnyEncryptedV3Column +> = new Map( + Object.values(v3Types).map((factory) => [ + factory('__probe__').getEqlType(), + factory, + ]), +) + +/** Every concrete `public.` string, derived from the eql/v3 factories. */ +export const EQL_V3_DOMAINS: ReadonlySet = new Set( + buildersByDomain.keys(), +) + +/** + * Drizzle passes `config.customTypeParams` from the builder into the processed + * PgColumn by reference — it is never cloned or serialized — so a `Symbol.for` + * key survives the whole builder→column path. Stashing the v3 builder there + * keeps recovery tied to the concrete column instance instead of a + * module-global name lookup. `Symbol.for` is registry-global, so a CJS/ESM + * duality of this module still resolves the same key. + */ +const EQL_V3_COLUMN_PARAM = Symbol.for('cipherstash:eqlv3Column') + +type EqlV3ColumnCarrier = Record + +function readBuilder( + carrier: EqlV3ColumnCarrier | undefined, +): AnyEncryptedV3Column | undefined { + return carrier?.[EQL_V3_COLUMN_PARAM] as AnyEncryptedV3Column | undefined +} + +function writeBuilder( + carrier: EqlV3ColumnCarrier | undefined, + builder: AnyEncryptedV3Column, +): void { + if (!carrier) return + carrier[EQL_V3_COLUMN_PARAM] = builder +} + +function getCarrier(column: unknown): EqlV3ColumnCarrier | undefined { + if (!column || typeof column !== 'object') return undefined + + const direct = column as EqlV3ColumnCarrier + if (readBuilder(direct)) return direct + + const maybeWithConfig = column as { + config?: { customTypeParams?: EqlV3ColumnCarrier } + } + return maybeWithConfig.config?.customTypeParams +} + +function getSqlType(column: unknown): string | undefined { + if (!column || typeof column !== 'object') return undefined + const columnAny = column as { + getSQLType?: () => unknown + dataType?: unknown + sqlName?: unknown + } + const sqlType = + typeof columnAny.getSQLType === 'function' + ? columnAny.getSQLType() + : undefined + if (typeof sqlType === 'string') return sqlType + const dt = columnAny.dataType + const domain = typeof dt === 'function' ? dt() : (columnAny.sqlName ?? dt) + return typeof domain === 'string' ? domain : undefined +} + +export function makeEqlV3Column(builder: C) { + const domain = builder.getEqlType() + const name = builder.getName() + + // What is stored/inserted/selected is the ENCRYPTED EQL v3 jsonb envelope + // (produced by `client.encrypt` / `bulkEncryptModels`), NOT the column's + // plaintext. So `data` is the envelope type — an insert takes an already- + // encrypted `Encrypted`, and a select yields one, ready for `decryptModel`. + const column = customType<{ data: Encrypted; driverData: string | null }>({ + dataType() { + return domain + }, + toDriver(value: Encrypted): string | null { + return v3ToDriver(value) + }, + fromDriver(value: string | object | null | undefined): Encrypted { + // A present jsonb value round-trips to an envelope; the driver only + // reaches here for non-null values, so the SQL-NULL branch is a safety + // net rather than a live path (the boundary cast covers it). + return v3FromDriver(value) as Encrypted + }, + })(name) + + writeBuilder(getCarrier(column), builder) + writeBuilder(column as unknown as EqlV3ColumnCarrier, builder) + return column +} + +export function getEqlV3Column( + columnName: string, + column: unknown, +): AnyEncryptedV3Column | undefined { + const stashed = readBuilder(getCarrier(column)) + if (stashed) return stashed + + const sqlType = getSqlType(column) + const builderFactory = sqlType ? buildersByDomain.get(sqlType) : undefined + return builderFactory?.(columnName) +} + +export function isEqlV3Column(column: unknown): boolean { + if (!column || typeof column !== 'object') return false + if (readBuilder(getCarrier(column))) return true + const sqlType = getSqlType(column) + return typeof sqlType === 'string' && EQL_V3_DOMAINS.has(sqlType) +} diff --git a/packages/stack/src/eql/v3/drizzle/index.ts b/packages/stack/src/eql/v3/drizzle/index.ts new file mode 100644 index 000000000..0eaf98302 --- /dev/null +++ b/packages/stack/src/eql/v3/drizzle/index.ts @@ -0,0 +1,8 @@ +export { v3FromDriver, v3ToDriver } from './codec.js' +export { getEqlV3Column, isEqlV3Column, makeEqlV3Column } from './column.js' +export { + createEncryptionOperatorsV3, + EncryptionOperatorError, +} from './operators.js' +export { extractEncryptionSchemaV3 } from './schema-extraction.js' +export { types } from './types.js' diff --git a/packages/stack/src/eql/v3/drizzle/operators.ts b/packages/stack/src/eql/v3/drizzle/operators.ts new file mode 100644 index 000000000..e5f927997 --- /dev/null +++ b/packages/stack/src/eql/v3/drizzle/operators.ts @@ -0,0 +1,527 @@ +import { + and, + asc, + Column, + desc, + exists, + is, + isNotNull, + isNull, + not, + notExists, + or, + type SQL, + type SQLWrapper, + sql, +} from 'drizzle-orm' +import type { PgTable } from 'drizzle-orm/pg-core' +import type { AuditConfig } from '@/encryption/operations/base-operation' +import type { AnyEncryptedV3Column, AnyV3Table } from '@/eql/v3' +import type { LockContext } from '@/identity' +import type { ColumnSchema } from '@/schema' +import { getEqlV3Column } from './column.js' +import { + extractEncryptionSchemaV3, + getDrizzleTableName, +} from './schema-extraction.js' +import { type ComparisonOp, type EqualityOp, v3Dialect } from './sql-dialect.js' + +const MAX_IN_ARRAY_CONCURRENCY = 4 + +/** + * The client capabilities this factory consumes: `encrypt`, and `bulkEncrypt` + * when the client offers it. Declared structurally — with maximally-permissive + * operands — so it is satisfied by the nominal `EncryptionClient`, by the + * `TypedEncryptionClient` that `EncryptionV3` returns (whatever its schema + * tuple), AND by a hand-rolled test double, none needing a cast. Typing the + * parameter to the nominal `TypedEncryptionClient` would reject a client + * built for a narrower schema tuple (it accepts fewer tables than + * `readonly AnyV3Table[]`); the structural surface sidesteps that variance. The + * factory resolves the column/table at runtime and encrypts through its own + * casts, so it relies on none of the client's per-column `encrypt` overloads. + * + * `bulkEncrypt` is optional so a `{ encrypt }`-only client stays valid; the + * list operators fall back to bounded-concurrency single encryption without it. + */ +type OperandEncryptionClient = { + encrypt( + plaintext: never, + opts: { table: AnyV3Table; column: AnyEncryptedV3Column }, + ): unknown + bulkEncrypt?( + plaintexts: never, + opts: { table: AnyV3Table; column: AnyEncryptedV3Column }, + ): unknown +} + +/** + * A dedicated error for v3 operator gating and operand-encryption failures, + * carrying the offending column/table/operator for diagnostics. + * + * INTENTIONAL FORK: this mirrors the v2 adapter's `EncryptionOperatorError` + * rather than sharing it. Unifying the two would couple `./drizzle` and + * `./eql/v3/drizzle` — two independently-versioned public entry points — so the + * duplication is deliberate, not an oversight. + */ +export class EncryptionOperatorError extends Error { + constructor( + message: string, + public readonly context?: { + columnName?: string + tableName?: string + operator?: string + }, + ) { + super(message) + this.name = 'EncryptionOperatorError' + } +} + +interface ColumnContext { + builder: AnyEncryptedV3Column + table: AnyV3Table + indexes: ColumnSchema['indexes'] + columnName: string + tableName: string +} + +export type EncryptionOperatorCallOpts = { + lockContext?: LockContext + audit?: AuditConfig +} + +type ChainableOperation = { + withLockContext(lockContext: LockContext): ChainableOperation + audit(config: AuditConfig): ChainableOperation + then: PromiseLike<{ + data?: unknown + failure?: { message: string } + }>['then'] +} + +async function mapWithConcurrency( + values: T[], + limit: number, + mapper: (value: T) => Promise, +): Promise { + const results = new Array(values.length) + let next = 0 + + async function worker(): Promise { + while (true) { + const index = next + next += 1 + if (index >= values.length) return + results[index] = await mapper(values[index]) + } + } + + await Promise.all( + Array.from({ length: Math.min(limit, values.length) }, () => worker()), + ) + return results +} + +/** + * Build v3-aware query operators (`eq`, `gte`, `contains`, `asc`, …) bound to an + * encryption `client`. Each comparison/containment operator AUTO-ENCRYPTS its + * plaintext operand into an EQL v3 query term before handing it to Drizzle, so + * callers pass plaintext and the emitted SQL compares encrypted values. Every + * operator also gates on the target column's capabilities and throws + * {@link EncryptionOperatorError} when the column can't answer the operator + * (e.g. ordering a non-`ore` column). + * + * @param client - anything that can `encrypt` — the nominal `EncryptionClient` + * or the `TypedEncryptionClient` from `EncryptionV3` (no cast needed). + * @param defaults - lock context / audit applied to every operand encryption + * unless a per-call override is supplied. + * + * @example + * ```typescript + * const ops = createEncryptionOperatorsV3(await EncryptionV3({ schemas: [users] })) + * await db.select().from(users).where(await ops.eq(users.email, 'a@b.com')) + * ``` + */ +export function createEncryptionOperatorsV3( + client: OperandEncryptionClient, + defaults: EncryptionOperatorCallOpts = {}, +) { + const tableCache = new WeakMap() + // Per-column context memo. `resolveContext` is value-independent, so caching + // by column identity makes `inArray`/`notInArray` build the context (and its + // deep-cloned match block) once for the whole list instead of once per value. + const contextCache = new WeakMap() + + function drizzleTableOf(column: SQLWrapper): PgTable | undefined { + return is(column, Column) + ? (column.table as PgTable | undefined) + : undefined + } + + function resolveContext(column: SQLWrapper, operator: string): ColumnContext { + const cached = contextCache.get(column) + if (cached) return cached + + const columnName = is(column, Column) ? column.name : 'unknown' + const builder = getEqlV3Column(columnName, column) + if (!builder) { + throw new EncryptionOperatorError( + `Operator "${operator}" requires an encrypted v3 column, but "${columnName}" is not one.`, + { columnName, operator }, + ) + } + + const drizzleTable = drizzleTableOf(column) + const tableName = getDrizzleTableName(drizzleTable) ?? 'unknown' + + let table = drizzleTable ? tableCache.get(drizzleTable) : undefined + if (!table && drizzleTable) { + table = extractEncryptionSchemaV3(drizzleTable) + tableCache.set(drizzleTable, table) + } + if (!table) { + throw new EncryptionOperatorError( + `Unable to resolve the encrypted table for column "${columnName}".`, + { columnName, operator }, + ) + } + + const context: ColumnContext = { + builder, + table, + indexes: builder.build().indexes, + columnName, + tableName, + } + contextCache.set(column, context) + return context + } + + /** + * Gate an operator on the column's indexes. `indexes` is a disjunction — any + * one of them grants the capability — so equality (`unique` OR `ore`) and the + * single-index gates share one rule and one diagnostic shape. + */ + function requireIndex( + ctx: ColumnContext, + indexes: readonly ('unique' | 'ore' | 'match')[], + operator: string, + capability: string, + ): void { + if (!indexes.some((index) => ctx.indexes[index])) { + throw new EncryptionOperatorError( + `Operator "${operator}" requires ${capability} on column "${ctx.columnName}" (domain ${ctx.builder.getEqlType()} does not support it).`, + { columnName: ctx.columnName, tableName: ctx.tableName, operator }, + ) + } + } + + const EQUALITY_INDEXES = ['unique', 'ore'] as const + const ORE_INDEXES = ['ore'] as const + const MATCH_INDEXES = ['match'] as const + + function applyOperationOptions( + op: ChainableOperation, + opts?: EncryptionOperatorCallOpts, + ): ChainableOperation { + const lockContext = opts?.lockContext ?? defaults.lockContext + const audit = opts?.audit ?? defaults.audit + const withLock = lockContext ? op.withLockContext(lockContext) : op + if (audit) withLock.audit(audit) + return withLock + } + + function requireNonNullOperand( + ctx: ColumnContext, + value: unknown, + operator: string, + ): void { + if (value == null) { + throw new EncryptionOperatorError( + `Operator "${operator}" cannot encrypt a null operand for column "${ctx.columnName}". Use isNull() or isNotNull() for NULL checks.`, + { + columnName: ctx.columnName, + tableName: ctx.tableName, + operator, + }, + ) + } + } + + function operandFailure( + ctx: ColumnContext, + operator: string, + reason: string, + ): EncryptionOperatorError { + return new EncryptionOperatorError( + `Failed to encrypt query operand for "${ctx.columnName}": ${reason}`, + { columnName: ctx.columnName, tableName: ctx.tableName, operator }, + ) + } + + async function encryptOperand( + ctx: ColumnContext, + value: unknown, + operator: string, + opts?: EncryptionOperatorCallOpts, + ): Promise { + requireNonNullOperand(ctx, value, operator) + + const result = await applyOperationOptions( + client.encrypt(value as never, { + table: ctx.table, + column: ctx.builder, + }) as unknown as ChainableOperation, + opts, + ) + if (result.failure) { + throw operandFailure(ctx, operator, result.failure.message) + } + return sql`${JSON.stringify(result.data)}` + } + + /** + * Encrypt a whole operand list. Prefers the client's `bulkEncrypt` — one FFI + * crossing for the entire list, rather than one per value — and falls back to + * bounded-concurrency single encryption for clients that don't expose it. + * + * `bulkEncrypt` is position-stable, so the returned terms align index-for- + * index with `values`; a response of a different length means the contract + * was violated and is rejected rather than silently truncating the predicate + * (which would widen an `inArray` or narrow a `notInArray`). + */ + async function encryptOperands( + ctx: ColumnContext, + values: unknown[], + operator: string, + opts?: EncryptionOperatorCallOpts, + ): Promise { + for (const value of values) requireNonNullOperand(ctx, value, operator) + + const bulkEncrypt = client.bulkEncrypt?.bind(client) + if (!bulkEncrypt) { + return mapWithConcurrency(values, MAX_IN_ARRAY_CONCURRENCY, (value) => + encryptOperand(ctx, value, operator, opts), + ) + } + + const result = await applyOperationOptions( + bulkEncrypt(values.map((plaintext) => ({ plaintext })) as never, { + table: ctx.table, + column: ctx.builder, + }) as unknown as ChainableOperation, + opts, + ) + if (result.failure) { + throw operandFailure(ctx, operator, result.failure.message) + } + + const encrypted = result.data as Array<{ data: unknown }> | undefined + if (!Array.isArray(encrypted) || encrypted.length !== values.length) { + throw operandFailure( + ctx, + operator, + `bulk encryption returned ${Array.isArray(encrypted) ? encrypted.length : 0} terms for ${values.length} values.`, + ) + } + return encrypted.map((term) => sql`${JSON.stringify(term.data)}`) + } + + const colSql = (column: SQLWrapper): SQL => sql`${column}` + + async function equality( + op: EqualityOp, + left: SQLWrapper, + right: unknown, + opts?: EncryptionOperatorCallOpts, + ): Promise { + const ctx = resolveContext(left, op) + requireIndex(ctx, EQUALITY_INDEXES, op, 'equality') + const enc = await encryptOperand(ctx, right, op, opts) + return v3Dialect.equality(op, colSql(left), enc) + } + + async function comparison( + op: ComparisonOp, + left: SQLWrapper, + right: unknown, + opts?: EncryptionOperatorCallOpts, + ): Promise { + const ctx = resolveContext(left, op) + requireIndex(ctx, ORE_INDEXES, op, 'order/range') + const enc = await encryptOperand(ctx, right, op, opts) + return v3Dialect.comparison(op, colSql(left), enc) + } + + async function range( + left: SQLWrapper, + min: unknown, + max: unknown, + negate: boolean, + operator: string, + opts?: EncryptionOperatorCallOpts, + ): Promise { + const ctx = resolveContext(left, operator) + requireIndex(ctx, ORE_INDEXES, operator, 'order/range') + // Independent operands — encrypt concurrently rather than paying two + // sequential round-trips to the crypto backend. + const [encMin, encMax] = await Promise.all([ + encryptOperand(ctx, min, operator, opts), + encryptOperand(ctx, max, operator, opts), + ]) + // `v3Dialect.range` is already parenthesised, so `NOT` binds to the whole + // conjunction without a wrapper here. + const condition = v3Dialect.range(colSql(left), encMin, encMax) + return negate ? sql`NOT ${condition}` : condition + } + + async function contains( + left: SQLWrapper, + right: unknown, + operator: string, + opts?: EncryptionOperatorCallOpts, + ): Promise { + const ctx = resolveContext(left, operator) + requireIndex(ctx, MATCH_INDEXES, operator, 'free-text search') + const enc = await encryptOperand(ctx, right, operator, opts) + return v3Dialect.contains(colSql(left), enc) + } + + async function inArrayOp( + left: SQLWrapper, + values: unknown[], + negate: boolean, + operator: string, + opts?: EncryptionOperatorCallOpts, + ): Promise { + const ctx = resolveContext(left, operator) + if (values.length === 0) { + throw new EncryptionOperatorError( + `Operator "${operator}" requires a non-empty list of values for column "${ctx.columnName}".`, + { columnName: ctx.columnName, tableName: ctx.tableName, operator }, + ) + } + // Gate and resolve the context once for the whole list, then encrypt it in + // a single crossing where the client supports it. + requireIndex(ctx, EQUALITY_INDEXES, operator, 'equality') + const op: EqualityOp = negate ? 'ne' : 'eq' + const encrypted = await encryptOperands(ctx, values, operator, opts) + const conditions = encrypted.map((enc) => + v3Dialect.equality(op, colSql(left), enc), + ) + const combined = negate ? and(...conditions) : or(...conditions) + return combined ?? (negate ? sql`true` : sql`false`) + } + + function orderTerm(column: SQLWrapper, operator: string): SQL { + const ctx = resolveContext(column, operator) + requireIndex(ctx, ORE_INDEXES, operator, 'order/range') + return v3Dialect.orderBy(colSql(column)) + } + + async function combine( + joiner: typeof and, + empty: SQL, + conditions: (SQL | SQLWrapper | Promise | undefined)[], + ): Promise { + const present = conditions.filter( + (c): c is SQL | SQLWrapper | Promise => c !== undefined, + ) + const resolved = await Promise.all(present) + return joiner(...resolved) ?? empty + } + + return { + /** Equality: `column = value`. Encrypts `r` and emits `eql_v3.eq`. + * Requires a `unique` or `ore` index on the column. */ + eq: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) => + equality('eq', l, r, opts), + /** Inequality: `column <> value`. Encrypts `r` and emits `eql_v3.neq`. + * Requires a `unique` or `ore` index on the column. */ + ne: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) => + equality('ne', l, r, opts), + /** Greater-than: `column > value`. Encrypts `r` and emits `eql_v3.gt`. + * Requires an `ore` (order/range) index. */ + gt: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) => + comparison('gt', l, r, opts), + /** Greater-than-or-equal: `column >= value`. Encrypts `r` and emits + * `eql_v3.gte`. Requires an `ore` (order/range) index. */ + gte: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) => + comparison('gte', l, r, opts), + /** Less-than: `column < value`. Encrypts `r` and emits `eql_v3.lt`. + * Requires an `ore` (order/range) index. */ + lt: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) => + comparison('lt', l, r, opts), + /** Less-than-or-equal: `column <= value`. Encrypts `r` and emits + * `eql_v3.lte`. Requires an `ore` (order/range) index. */ + lte: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) => + comparison('lte', l, r, opts), + /** Inclusive range `min <= column <= max`. Encrypts both bounds + * concurrently. Requires an `ore` (order/range) index. */ + between: ( + l: SQLWrapper, + min: unknown, + max: unknown, + opts?: EncryptionOperatorCallOpts, + ) => range(l, min, max, false, 'between', opts), + /** Negated inclusive range `NOT (min <= column <= max)`. Encrypts both + * bounds concurrently. Requires an `ore` (order/range) index. */ + notBetween: ( + l: SQLWrapper, + min: unknown, + max: unknown, + opts?: EncryptionOperatorCallOpts, + ) => range(l, min, max, true, 'notBetween', opts), + /** Free-text containment: emits `eql_v3.contains` over the encrypted match + * term. Encrypts `r`. Requires a `match` (free-text search) index. */ + contains: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) => + contains(l, r, 'contains', opts), + /** Membership: ORs one encrypted `eq` term per value. The whole list is + * encrypted in one `bulkEncrypt` crossing where the client supports it, + * otherwise concurrency-bounded. Rejects an empty list; requires a + * `unique` or `ore` index. */ + inArray: ( + l: SQLWrapper, + values: unknown[], + opts?: EncryptionOperatorCallOpts, + ) => inArrayOp(l, values, false, 'inArray', opts), + /** Non-membership: ANDs one encrypted `ne` term per value. The whole list + * is encrypted in one `bulkEncrypt` crossing where the client supports it, + * otherwise concurrency-bounded. Rejects an empty list; requires a + * `unique` or `ore` index. */ + notInArray: ( + l: SQLWrapper, + values: unknown[], + opts?: EncryptionOperatorCallOpts, + ) => inArrayOp(l, values, true, 'notInArray', opts), + /** Ascending order by the encrypted order term (`eql_v3.ord_term`). + * Synchronous (no operand to encrypt). Requires an `ore` index. */ + asc: (c: SQLWrapper) => asc(orderTerm(c, 'asc')), + /** Descending order by the encrypted order term (`eql_v3.ord_term`). + * Synchronous (no operand to encrypt). Requires an `ore` index. */ + desc: (c: SQLWrapper) => desc(orderTerm(c, 'desc')), + /** Conjunction of the given conditions, awaiting any async operands and + * dropping `undefined`. Empty input resolves to `true`. */ + and: (...conds: (SQL | SQLWrapper | Promise | undefined)[]) => + combine(and, sql`true`, conds), + /** Disjunction of the given conditions, awaiting any async operands and + * dropping `undefined`. Empty input resolves to `false`. */ + or: (...conds: (SQL | SQLWrapper | Promise | undefined)[]) => + combine(or, sql`false`, conds), + /** Drizzle's `isNull`, re-exported unchanged — `column IS NULL` needs no + * encryption and works on any (nullable) encrypted column. */ + isNull, + /** Drizzle's `isNotNull`, re-exported unchanged — `column IS NOT NULL` + * needs no encryption. */ + isNotNull, + /** Drizzle's `not`, re-exported unchanged — negates an already-built + * (encrypted) predicate. Safe over any operator here, including `between`, + * whose fragment is self-parenthesising. */ + not, + /** Drizzle's `exists`, re-exported unchanged — for correlated subqueries. */ + exists, + /** Drizzle's `notExists`, re-exported unchanged — for correlated + * subqueries. */ + notExists, + } +} diff --git a/packages/stack/src/eql/v3/drizzle/schema-extraction.ts b/packages/stack/src/eql/v3/drizzle/schema-extraction.ts new file mode 100644 index 000000000..623804ba9 --- /dev/null +++ b/packages/stack/src/eql/v3/drizzle/schema-extraction.ts @@ -0,0 +1,50 @@ +import type { PgTable } from 'drizzle-orm/pg-core' +import { + type AnyEncryptedV3Column, + type AnyV3Table, + encryptedTable, +} from '@/eql/v3' +import { getEqlV3Column } from './column.js' + +/** Drizzle stashes the SQL table name on this well-known symbol key. */ +const DRIZZLE_NAME = Symbol.for('drizzle:Name') + +/** + * Read the SQL table name Drizzle stashes on a `pgTable`. Returns `undefined` + * for a non-object or a table not built with `pgTable()`. Shared by + * {@link extractEncryptionSchemaV3} and the operator factory so the + * symbol-key introspection lives in exactly one place. + */ +export function getDrizzleTableName(table: unknown): string | undefined { + if (!table || typeof table !== 'object') return undefined + const name = (table as Record)[DRIZZLE_NAME] + return typeof name === 'string' ? name : undefined +} + +export function extractEncryptionSchemaV3(table: PgTable): AnyV3Table { + const tableName = getDrizzleTableName(table) + if (!tableName) { + throw new Error( + 'Unable to read table name from Drizzle table. Use a table created with pgTable().', + ) + } + + const columns: Record = {} + for (const [property, column] of Object.entries(table)) { + if (typeof column !== 'object' || column === null) continue + const columnName = + 'name' in column && typeof column.name === 'string' + ? column.name + : property + const builder = getEqlV3Column(columnName, column) + if (builder) columns[property] = builder + } + + if (Object.keys(columns).length === 0) { + throw new Error( + `No encrypted v3 columns found in table "${tableName}". Declare columns with the v3 drizzle \`types\` namespace.`, + ) + } + + return encryptedTable(tableName, columns) +} diff --git a/packages/stack/src/eql/v3/drizzle/sql-dialect.ts b/packages/stack/src/eql/v3/drizzle/sql-dialect.ts new file mode 100644 index 000000000..d501eff87 --- /dev/null +++ b/packages/stack/src/eql/v3/drizzle/sql-dialect.ts @@ -0,0 +1,47 @@ +import { type SQL, sql } from 'drizzle-orm' + +export type EqualityOp = 'eq' | 'ne' +export type ComparisonOp = 'gt' | 'gte' | 'lt' | 'lte' + +/** + * The schema the EQL v3 comparison functions live in. The concrete `public.*` + * domains are derived from the column factories (see `column.ts`); the + * functions that operate on them are namespaced separately, so they get their + * own single source of truth. + */ +export const EQL_V3_FN_SCHEMA = 'eql_v3' + +const fn = (name: string): SQL => sql.raw(`${EQL_V3_FN_SCHEMA}.${name}`) + +export const v3Dialect = { + equality(op: EqualityOp, left: SQL, enc: SQL): SQL { + return sql`${fn(op === 'eq' ? 'eq' : 'neq')}(${left}, ${enc}::jsonb)` + }, + + comparison(op: ComparisonOp, left: SQL, enc: SQL): SQL { + return sql`${fn(op)}(${left}, ${enc}::jsonb)` + }, + + /** + * Inclusive range, emitted as a SELF-CONTAINED parenthesised conjunction. + * + * The parentheses are load-bearing: this is the only helper returning more + * than one function call, and Drizzle's `not` renders `not ${condition}` + * without adding any of its own. Postgres binds NOT tighter than AND, so a + * bare fragment would make `not(between(col, x, y))` parse as + * `(NOT gte(col, x)) AND lte(col, y)` — rows below the lower bound rather + * than the complement of the range. Parenthesising here makes every + * composition safe instead of asking each caller to remember. + */ + range(left: SQL, min: SQL, max: SQL): SQL { + return sql`(${fn('gte')}(${left}, ${min}::jsonb) AND ${fn('lte')}(${left}, ${max}::jsonb))` + }, + + contains(left: SQL, enc: SQL): SQL { + return sql`${fn('contains')}(${left}, ${enc}::jsonb)` + }, + + orderBy(left: SQL): SQL { + return sql`${fn('ord_term')}(${left})` + }, +} diff --git a/packages/stack/src/eql/v3/drizzle/types.ts b/packages/stack/src/eql/v3/drizzle/types.ts new file mode 100644 index 000000000..b52fe83f3 --- /dev/null +++ b/packages/stack/src/eql/v3/drizzle/types.ts @@ -0,0 +1,19 @@ +import { types as v3Types } from '@/eql/v3' +import { makeEqlV3Column } from './column.js' + +type V3Types = typeof v3Types + +/** + * Drizzle-native mirror of `@/eql/v3`'s `types` namespace. Each PascalCase + * factory returns a Drizzle column wrapping the matching concrete v3 builder. + */ +export const types = Object.fromEntries( + Object.entries(v3Types).map(([name, factory]) => [ + name, + (columnName: string) => makeEqlV3Column(factory(columnName)), + ]), +) as { + [K in keyof V3Types]: ( + name: string, + ) => ReturnType>> +} diff --git a/packages/stack/src/eql/v3/types.ts b/packages/stack/src/eql/v3/types.ts index 2ae0a1c70..ecbdea3e2 100644 --- a/packages/stack/src/eql/v3/types.ts +++ b/packages/stack/src/eql/v3/types.ts @@ -103,9 +103,10 @@ import { * }) * ``` * - * `types.TextSearch` keeps the chainable `.freeTextSearch(opts)` tuner (the - * only capability-bearing chain — every other domain is fully described by its - * type). + * `types.TextSearch` is fully described by its type — like every other domain, + * it has no capability-bearing or tuning chain. Its match index is always + * emitted with the default configuration; run a free-text query by passing + * `queryType: 'freeTextSearch'` to `encryptQuery`. */ export const types = { // integer diff --git a/packages/stack/tsup.config.ts b/packages/stack/tsup.config.ts index 2380c8c49..0a342ae0a 100644 --- a/packages/stack/tsup.config.ts +++ b/packages/stack/tsup.config.ts @@ -16,6 +16,7 @@ export default defineConfig([ 'src/secrets/index.ts', 'src/schema/index.ts', 'src/eql/v3/index.ts', + 'src/eql/v3/drizzle/index.ts', 'src/drizzle/index.ts', 'src/dynamodb/index.ts', 'src/supabase/index.ts', diff --git a/packages/stack/vitest.config.ts b/packages/stack/vitest.config.ts index f27fc47cc..2f8acabec 100644 --- a/packages/stack/vitest.config.ts +++ b/packages/stack/vitest.config.ts @@ -37,6 +37,16 @@ export default defineConfig({ // out of scope (tracked as a follow-up). Run via the `test:types` script // with `--typecheck.only` so the runtime suites do NOT also execute. tsconfig: './tsconfig.typecheck.json', + // Type coverage lives in `*.test-d.ts`. M1 (the factory rejecting the + // `TypedEncryptionClient` that `EncryptionV3` returns) slipped through + // because the runtime `*.test.ts` suites are not typechecked. Rather than + // widen to those files — the drizzle-v3 suites are pervasively loose-typed + // by design (dynamic domain matrix: `as never` tables, untyped mock + // introspection, `readonly` sample tuples) so a wholesale widen surfaces + // dozens of pre-existing, unrelated errors AND risks exposing the + // wasm-inline errors via their imports — M1 and the A3 envelope contract + // are locked directly in `drizzle-v3/operators.test-d.ts` and + // `drizzle-v3/types.test-d.ts`, which this glob already covers. include: ['__tests__/**/*.test-d.ts'], }, },