feat(stack,cli): EQL v3 typed schema, typed client, Supabase/Drizzle integrations on eql-3.0.0 + protect-ffi 0.29#535
feat(stack,cli): EQL v3 typed schema, typed client, Supabase/Drizzle integrations on eql-3.0.0 + protect-ffi 0.29#535tobyhede wants to merge 142 commits into
Conversation
🦋 Changeset detectedLatest commit: 9229bad The changes in this PR will be included in the next version bump. This PR includes changesets to release 11 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR adds a v3 ChangesEQL v3 text_search schema DSL and client type widening
CLI dotenv loading and non-PTY test helper
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/stack/src/types.ts (1)
111-113: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valuePossible redundant union arm.
If
EncryptedColumnalready has a publicgetEqlType()method and no private/protected members forcing nominal typing, the explicitEncryptedColumnarm is structurally subsumed byBuildableColumn & { getEqlType(): string }and could be dropped for simplicity. Not a functional issue either way; only worth simplifying if confirmed redundant.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/stack/src/types.ts` around lines 111 - 113, The BuildableQueryColumn union appears to have a redundant arm if EncryptedColumn already satisfies BuildableColumn & { getEqlType(): string } structurally. Check the EncryptedColumn type and, if it does not rely on nominal typing via private/protected members, simplify the BuildableQueryColumn alias in types.ts by removing the explicit EncryptedColumn union member and keeping only the shared structural form.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/stack/src/types.ts`:
- Around line 111-113: The BuildableQueryColumn union appears to have a
redundant arm if EncryptedColumn already satisfies BuildableColumn & {
getEqlType(): string } structurally. Check the EncryptedColumn type and, if it
does not rely on nominal typing via private/protected members, simplify the
BuildableQueryColumn alias in types.ts by removing the explicit EncryptedColumn
union member and keeping only the shared structural form.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 01405e99-c5e9-4153-8af3-10759fd8ebbd
📒 Files selected for processing (16)
.changeset/eql-v3-text-search.md.github/workflows/tests.ymldocs/superpowers/plans/2026-06-30-eql-v3-text-search-schema-plan.mddocs/superpowers/specs/2026-06-30-eql-v3-text-search-schema-design.mdpackages/stack/__tests__/schema-v3.test-d.tspackages/stack/__tests__/schema-v3.test.tspackages/stack/package.jsonpackages/stack/src/encryption/helpers/infer-index-type.tspackages/stack/src/encryption/operations/bulk-encrypt.tspackages/stack/src/encryption/operations/encrypt.tspackages/stack/src/schema/index.tspackages/stack/src/schema/v3/index.tspackages/stack/src/types.tspackages/stack/tsconfig.typecheck.jsonpackages/stack/tsup.config.tspackages/stack/vitest.config.ts
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/cli/tests/helpers/run.ts (1)
44-87: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low valueNo timeout/kill safeguard for a hung child.
If the spawned CLI process hangs (e.g., waiting on unexpected input despite
stdio: ['ignore', ...]), nothing here kills it — the test will eventually time out via vitest, but the orphaned child process keeps running. Consider an optional timeout that callschild.kill()and rejects.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/tests/helpers/run.ts` around lines 44 - 87, The run helper currently waits indefinitely for the spawned CLI in run() and only resolves on close, so a hung child can outlive the test. Update packages/cli/tests/helpers/run.ts by adding an optional timeout to RunOptions and wiring it in run() to call child.kill() and reject if the process does not exit in time, while preserving the existing stdout/stderr capture and cleanup in the child.on('close') path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/cli/tests/helpers/run.ts`:
- Around line 44-87: The run helper currently waits indefinitely for the spawned
CLI in run() and only resolves on close, so a hung child can outlive the test.
Update packages/cli/tests/helpers/run.ts by adding an optional timeout to
RunOptions and wiring it in run() to call child.kill() and reject if the process
does not exit in time, while preserving the existing stdout/stderr capture and
cleanup in the child.on('close') path.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 8faee085-09ef-4411-822e-50addd54c10c
📒 Files selected for processing (5)
packages/cli/src/bin/main.tspackages/cli/tests/e2e/runner-aware-help.e2e.test.tspackages/cli/tests/e2e/smoke.e2e.test.tspackages/cli/tests/helpers/run.tspackages/stack/src/schema/v3/index.ts
✅ Files skipped from review due to trivial changes (1)
- packages/cli/tests/e2e/smoke.e2e.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/stack/src/schema/v3/index.ts
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/stack/src/schema/v3/index.ts (1)
460-470: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winSnapshot nested
freeTextSearch()options when you store them.Lines 465-466 keep
opts.tokenizerandopts.token_filtersby reference, so mutating the caller’s options object after configuration silently changes this builder’s laterbuild()output. The rest of this class is explicitly avoiding shared nested state, so this should clone on write too.Suggested fix
freeTextSearch(opts?: MatchIndexOpts): this { // A fresh defaults object per call supplies the `?? ` fallbacks, so no // nested default object is ever shared into `this.matchOpts` by reference. const defaults = defaultMatchOpts() this.matchOpts = { - tokenizer: opts?.tokenizer ?? defaults.tokenizer, - token_filters: opts?.token_filters ?? defaults.token_filters, + tokenizer: opts?.tokenizer + ? { ...opts.tokenizer } + : { ...defaults.tokenizer }, + token_filters: opts?.token_filters + ? opts.token_filters.map((f) => ({ ...f })) + : defaults.token_filters.map((f) => ({ ...f })), k: opts?.k ?? defaults.k, m: opts?.m ?? defaults.m, include_original: opts?.include_original ?? defaults.include_original, } return this🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/stack/src/schema/v3/index.ts` around lines 460 - 470, The freeTextSearch method in the v3 schema builder is still storing nested options by reference, so later mutations to the caller’s MatchIndexOpts can leak into build() output. Update freeTextSearch in packages/stack/src/schema/v3/index.ts to clone the tokenizer and token_filters values when assigning this.matchOpts, matching the class’s existing clone-on-write approach and avoiding shared nested state.
🧹 Nitpick comments (6)
docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md (3)
510-510: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueFix redundant phrasing.
"Repeat the same exact pattern" → "Repeat the same pattern" or "Repeat this exact pattern".
- Repeat the same exact pattern for: + Repeat the same pattern for:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md` at line 510, Update the phrasing in the plans document to remove redundancy: change the “Repeat the same exact pattern for:” text to either “Repeat the same pattern for:” or “Repeat this exact pattern for:”. Locate the sentence in the document section containing that exact phrase and keep the rest unchanged.Source: Linters/SAST tools
117-117: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider rewording for readability.
Three successive list items begin with "Schemas with". While this is a list format where parallelism is expected, consider varying the structure if the static analysis tool flagged it as an issue.
- - Schemas with required `hm` support equality. - - Schemas with required `ob` support order/range. - - Schemas with required `bf` support free-text search. + - `hm` required → equality support. + - `ob` required → order/range support. + - `bf` required → free-text search support.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md` at line 117, Reword the list item about storage-only schemas to avoid repeating the same “Schemas with” sentence pattern as the surrounding bullets. Update the wording in the schema documentation section so it still conveys that schemas containing only v, i, and c are storage-only, but uses a different sentence structure for readability and to satisfy the static analysis warning.Source: Linters/SAST tools
27-27: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider rewording for readability.
Three successive sentences beginning with "In" - though this appears to be in the file structure list where parallelism is intentional. Given the context is a structured plan document, this is acceptable but could be tightened.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md` at line 27, The plan document wording is a bit repetitive in the file-structure list, where several consecutive bullets/sentences start with the same “In” pattern. Rephrase the affected entries in the schema-plan section to improve readability while preserving the parallel structure, keeping the “BuildableTable” bullet clear and concise.Source: Linters/SAST tools
packages/stack/scripts/install-eql-v3.ts (1)
1-1: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueBare
dotenv/configre-introduces the banner noise this cohort suppresses elsewhere.The CLI entrypoint now loads env files with
config({ path: '.env.local', quiet: true })specifically to avoid dotenv v17's injected-env banner. This script importsdotenv/configdirectly with no options, so it will still print that banner whenever it runs (e.g. in CI logs).♻️ Suggested fix
-import 'dotenv/config' +import { config } from 'dotenv' import postgres from 'postgres' import { installEqlV3IfNeeded } from '../__tests__/helpers/eql-v3' + +config({ quiet: true })🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/stack/scripts/install-eql-v3.ts` at line 1, The install-eql-v3 script is importing dotenv in a way that re-enables the noisy injected-env banner. Update the startup env loading in this script to use the same explicit dotenv config pattern as the CLI entrypoint, with a fixed .env.local path and quiet enabled, and remove the bare dotenv/config import so the script stays silent in CI. Reference the script’s top-level env bootstrap in install-eql-v3.ts.packages/stack/__tests__/schema-v3.test.ts (1)
2-3: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExercise this failure through a public API instead of
resolveIndexType.This test now imports
@/encryption/helpers/infer-index-typedirectly and asserts the helper’s internal error strings, which makes the suite brittle to refactors inside the implementation rather than the supported contract. Please move this misuse coverage to the public entry point that surfaces the same runtime failure. As per coding guidelines, "Prefer testing via public API; avoid reaching into private internals in tests".Also applies to: 661-677
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/stack/__tests__/schema-v3.test.ts` around lines 2 - 3, The test is reaching into the private `resolveIndexType` helper and asserting its internal error strings, which makes it brittle. Update `packages/stack/__tests__/schema-v3.test.ts` to remove the direct `@/encryption/helpers/infer-index-type` import and exercise the same failure through the public `encryptConfigSchema`/`encryptedColumn` API instead. Keep the coverage for the misuse case, but assert the runtime failure surfaced by the supported schema entry point rather than helper internals.Source: Coding guidelines
packages/stack/__tests__/schema-v3.test-d.ts (1)
229-301: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a model-inference case with aliased column names.
This file already proves v3 domain inference for aliased builders like
createdAt→created_at, but theencryptModel/bulkEncryptModelsacceptance cases only cover same-name keys. One typed assertion for an aliasedencryptedTimestamptzColumn('created_at')model field would protect the exact v3 field mapping this PR is widening.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/stack/__tests__/schema-v3.test-d.ts` around lines 229 - 301, Add a typed model-inference test for aliased v3 encrypted columns, since the current `encryptModel` and `bulkEncryptModels` cases only cover same-name fields. Update the `schema-v3.test-d.ts` assertions near the existing `encryptModel`/`bulkEncryptModels` checks to include a model using `encryptedTimestamptzColumn('created_at')` with an aliased property name. Verify the inferred `EncryptionClient.encryptModel` and/or `bulkEncryptModels` result type maps the aliased field correctly while still preserving unrelated fields.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md`:
- Around line 63-65: The task steps currently hardcode a developer-specific
absolute path in the read-only references, which will not work for other
environments. Update the instructions in the plan to use repository-relative
paths or an environment-agnostic placeholder, and if needed mention that the
base path must be configured by the user; keep the references to the
inventory.rs and schema/v3/*.json locations clear without embedding a local
machine path.
In `@packages/stack/__tests__/helpers/eql-v3.ts`:
- Around line 28-41: The advisory lock handling in installEqlV3IfNeeded is using
separate sql calls, so the lock and unlock may run on different pooled
connections. Update the function to run the entire check/install/unlock flow on
a reserved connection via sql.reserve(), or replace
pg_advisory_lock/pg_advisory_unlock with pg_advisory_xact_lock if the EQL v3
install path is transaction-safe, and keep the existing hasEqlV3TextSearch and
eqlV3Sql execution logic inside that reserved scope.
In `@packages/stack/__tests__/schema-v3-pg.test.ts`:
- Around line 133-149: The cleanup hooks in the schema-v3-pg test only remove
rows from protect_ci_v3_text_search, so the typed-domain fixture data in
protect_ci_v3_typed_domains is left behind. Update the existing beforeEach and
afterAll hooks in schema-v3-pg.test.ts to also delete rows for the typed-domain
table using the same TEST_RUN_ID guard, alongside the current cleanup logic. Use
the existing beforeEach, afterAll, and sql cleanup blocks as the place to add
the matching protect_ci_v3_typed_domains deletion.
In `@packages/stack/src/types.ts`:
- Around line 151-183: The public BuildableTable shape is too weak for the
encryption inference used by encryptModel() and bulkEncryptModels(), so
structurally accepted tables lose the literal column keys needed by
EncryptedFromBuildableTable. Fix this by either adding the column map
brand/_columnType to the BuildableTable contract and keeping
BuildableTableColumns aligned with it, or by narrowing the affected APIs/types
back to the branded table builder type so the return type reflects encrypted
fields correctly.
---
Outside diff comments:
In `@packages/stack/src/schema/v3/index.ts`:
- Around line 460-470: The freeTextSearch method in the v3 schema builder is
still storing nested options by reference, so later mutations to the caller’s
MatchIndexOpts can leak into build() output. Update freeTextSearch in
packages/stack/src/schema/v3/index.ts to clone the tokenizer and token_filters
values when assigning this.matchOpts, matching the class’s existing
clone-on-write approach and avoiding shared nested state.
---
Nitpick comments:
In `@docs/superpowers/plans/2026-07-01-eql-v3-typed-schema.md`:
- Line 510: Update the phrasing in the plans document to remove redundancy:
change the “Repeat the same exact pattern for:” text to either “Repeat the same
pattern for:” or “Repeat this exact pattern for:”. Locate the sentence in the
document section containing that exact phrase and keep the rest unchanged.
- Line 117: Reword the list item about storage-only schemas to avoid repeating
the same “Schemas with” sentence pattern as the surrounding bullets. Update the
wording in the schema documentation section so it still conveys that schemas
containing only v, i, and c are storage-only, but uses a different sentence
structure for readability and to satisfy the static analysis warning.
- Line 27: The plan document wording is a bit repetitive in the file-structure
list, where several consecutive bullets/sentences start with the same “In”
pattern. Rephrase the affected entries in the schema-plan section to improve
readability while preserving the parallel structure, keeping the
“BuildableTable” bullet clear and concise.
In `@packages/stack/__tests__/schema-v3.test-d.ts`:
- Around line 229-301: Add a typed model-inference test for aliased v3 encrypted
columns, since the current `encryptModel` and `bulkEncryptModels` cases only
cover same-name fields. Update the `schema-v3.test-d.ts` assertions near the
existing `encryptModel`/`bulkEncryptModels` checks to include a model using
`encryptedTimestamptzColumn('created_at')` with an aliased property name. Verify
the inferred `EncryptionClient.encryptModel` and/or `bulkEncryptModels` result
type maps the aliased field correctly while still preserving unrelated fields.
In `@packages/stack/__tests__/schema-v3.test.ts`:
- Around line 2-3: The test is reaching into the private `resolveIndexType`
helper and asserting its internal error strings, which makes it brittle. Update
`packages/stack/__tests__/schema-v3.test.ts` to remove the direct
`@/encryption/helpers/infer-index-type` import and exercise the same failure
through the public `encryptConfigSchema`/`encryptedColumn` API instead. Keep the
coverage for the misuse case, but assert the runtime failure surfaced by the
supported schema entry point rather than helper internals.
In `@packages/stack/scripts/install-eql-v3.ts`:
- Line 1: The install-eql-v3 script is importing dotenv in a way that re-enables
the noisy injected-env banner. Update the startup env loading in this script to
use the same explicit dotenv config pattern as the CLI entrypoint, with a fixed
.env.local path and quiet enabled, and remove the bare dotenv/config import so
the script stays silent in CI. Reference the script’s top-level env bootstrap in
install-eql-v3.ts.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 4aa02b84-db93-4a9b-aa84-d185e2c884d9
📒 Files selected for processing (25)
.changeset/eql-v3-typed-schema.mddocs/query-api-walkthrough.mddocs/superpowers/plans/2026-07-01-eql-v3-typed-schema.mdpackages/stack/__tests__/cjs-require.test.tspackages/stack/__tests__/fixtures/eql-v3/cipherstash-encrypt-v3.sqlpackages/stack/__tests__/helpers/eql-v3.tspackages/stack/__tests__/helpers/stub-auth-wasm-inline.tspackages/stack/__tests__/helpers/stub-protect-ffi-wasm-inline.tspackages/stack/__tests__/schema-v3-client.test.tspackages/stack/__tests__/schema-v3-pg.test.tspackages/stack/__tests__/schema-v3.test-d.tspackages/stack/__tests__/schema-v3.test.tspackages/stack/__tests__/wasm-inline-column-name.test.tspackages/stack/package.jsonpackages/stack/scripts/install-eql-v3.tspackages/stack/src/encryption/helpers/infer-index-type.tspackages/stack/src/encryption/helpers/model-helpers.tspackages/stack/src/encryption/index.tspackages/stack/src/encryption/operations/bulk-encrypt-models.tspackages/stack/src/encryption/operations/encrypt-model.tspackages/stack/src/encryption/operations/encrypt-query.tspackages/stack/src/encryption/operations/encrypt.tspackages/stack/src/schema/v3/index.tspackages/stack/src/types.tspackages/stack/vitest.config.ts
✅ Files skipped from review due to trivial changes (3)
- docs/query-api-walkthrough.md
- packages/stack/tests/helpers/stub-protect-ffi-wasm-inline.ts
- .changeset/eql-v3-typed-schema.md
🚧 Files skipped from review as they are similar to previous changes (4)
- packages/stack/tests/wasm-inline-column-name.test.ts
- packages/stack/src/encryption/helpers/infer-index-type.ts
- packages/stack/package.json
- packages/stack/src/encryption/operations/encrypt.ts
b54e6d4 to
ed78233
Compare
Column builders are copied onto the EncryptedTable instance for accessor access (users.email). A column named build/tableName/columnBuilders/ _columnType would silently overwrite that member — worst case a 'build' column breaks buildEncryptConfig's tb.build() call at runtime. Throw a clear error at table-definition time instead. Scoped to v3; v2 retains its existing behavior. Found by CodeRabbit review.
…ntry The wasm-inline encrypt entry typed opts.column as the widened structural BuildableColumn, but getColumnName still gated on instanceof EncryptedColumn || EncryptedField and threw for a v3 EncryptedTextSearchColumn — a runtime break the type promise hid. Resolve the name structurally (typeof getName) so v3 columns round-trip through WasmEncryptionClient.encrypt(); still throws for non-builder JS input. getColumnName is the only instanceof gate on this path; the rest reads table.tableName structurally. Adds wasm-inline-column-name.test.ts exercising the seam (v2 column/field + v3 column + non-builder). Like its sibling wasm-inline-normalize.test.ts the suite cannot load in environments missing the @cipherstash/protect-ffi /wasm-inline dep subpath.
Config tables are keyed by name, so two tables with the same tableName silently dropped the earlier one. Add a v3-only additive guard that throws on a duplicate (Object.hasOwn). v2's buildEncryptConfig keeps its existing silent-overwrite behavior (no-v2-change constraint).
The RESERVED_TABLE_KEYS guard only covered own members (build, tableName, columnBuilders, _columnType), so a column named constructor/toString/valueOf/ hasOwnProperty was assigned as an own property, shadowing the Object.prototype member. Add an `in` check (isReservedTableKey) so any prototype-chain member is also rejected, keeping the table object well-behaved for reflection.
…freeTextSearch' for match A v3 text_search column emits unique+ore+match, and shared index inference picks by priority unique > match > ore. So encryptQuery without an explicit queryType builds an EQUALITY term (via unique) — a substring matches nothing. Document on EncryptedTextSearchColumn + encryptedTextSearchColumn that callers must pass queryType:'freeTextSearch' (FFI 'match') for free-text queries. Addresses review finding #2 (naming footgun; doc-only, no runtime change).
Add table-driven runtime tests for all 40 EQL v3 domain builders (name, eqlType, capabilities, config, queryability) plus type-level tests for nominal domain distinctness, InferPlaintext mapping, queryability of BuildableQueryColumn, and v3/v2 model inference.
The regular filter path splits an `in` array and encrypts each element (query-builder.ts:533). The .or() path had no such case: it pushed ONE term whose value was the whole array. `JsPlaintext` admits arrays, so nothing rejected it; after substitution the value was a single string, the `(a,b)` list form was lost, and the filter could never match. Silent, fails closed, pre-existing in v2. - Collapse the string/structured or-term loops, which differed only in their `source` tag, so the in-split lands once for both. - Widen the or TermMapping key with an optional `inIndex`, so a per-element value never collides with a whole-condition value. - `substituteOrValue` rebuilds the array element-by-element, letting `formatOrValue` re-emit the `(a,b)` list form.
The hand-written supabase v3 wire tests exercised only 5 of the catalog's 39 domains; the whole numeric family (smallint/numeric/real/double/date/ bigint) never reached `EncryptedQueryBuilderV3Impl` at all, and nothing guarded that number. `supabase-v3-matrix.test.ts` reuses the compile-enforced `V3_MATRIX`, so adding a domain to the SDK yields a Supabase wire assertion for free — or fails to compile until it has a catalog row. Tiers are derived from `indexes`, exactly as `drizzle-v3/operators-live-pg.test.ts` derives them, rather than from `capabilities` (which is what the adapter's guard reads): deriving from the other field turns a future capability/index divergence into a visible tier mismatch instead of a test that agrees with itself. The tier counts are asserted, so a domain silently dropping out of a tier turns a test red rather than quietly shrinking an `it.each`. The storage-only tier asserts the capability-guard MESSAGE, not merely the 500 status, and takes `samples[0]` explicitly: after fd33aad a null operand skips encryption entirely and never reaches the guard, so a status-only assertion on a future null sample would pass for the wrong reason. Also extracts the mock encryption/supabase clients into `__tests__/helpers/supabase-mock.ts` so both suites drive the adapter through the same doubles. 24784e2 committed a `supabase-v3-builder.test.ts` that already imports this module without adding it; HEAD does not currently resolve that import. `fakeEnvelope` now normalizes `bigint` the way it already normalized `Date`: the v3 filter path carries `pt` through `JSON.stringify`, which throws on a BigInt. A real envelope only ever holds strings, so this is a mock artifact, not a product bug — but the four `public.bigint*` domains' samples hit it. Finally, the "35 domains" comments in the v3 matrix live suites were stale; the four bigint domains brought the catalog to 39.
Under the shipped grants, EVERY encrypted filter the supabase v3 adapter
emits fails for `anon` and `authenticated` with
permission denied for schema eql_v3_internal
`supabasePermissionsSql` granted exactly one schema, `eql_v3`. But the
public entry points the query path calls — `eql_v3.eq_term` (26 of 27
overloads), `ord_term` (38/38) and `match_term` (4/4) — are SECURITY
INVOKER and qualify `eql_v3_internal.*` by name in their bodies. Postgres
resolves those names with the CALLER's privileges and checks schema USAGE
at name resolution, so the call never reaches the function. `=` routes
through eq_term, `>=` through ord_term, `@>`/`cs` through match_term: there
is no encrypted operation that survives. The default PUBLIC EXECUTE on
functions means USAGE is the only real barrier; EXECUTE is granted anyway so
an install into a database that revoked it from PUBLIC still works.
`eql_v2` has no internal schema and is unaffected.
Fixes a second defect found on the way: `grantSupabasePermissions` rebuilt
the block from `supabasePermissionsSql(schemaNameFor(eqlVersion))` rather
than using the exported constant, so this fix would have reached the
generated Supabase migration and NOT the database `stash db install` writes
to. Both now go through `supabaseGrantsFor(eqlVersion)`, and the installer
test pins the executed string to the exported constant.
The grants SQL moves to an import-free `installer/grants.ts` so the live
proof in @cipherstash/stack can assert against the EXACT shipped strings;
`stash` already depends on @cipherstash/stack, so a dependency the other way
would be a cycle, and a local copy would drift.
`supabase-v3-grants-pg.test.ts` runs on the existing DATABASE_URL-only gate.
It needs no CipherStash credentials: `eql_v3_internal.ore_block_256('{}')`
fails on PERMISSION for a caller that cannot resolve the schema and on DATA
for one that can, and that distinction is the whole test. It revokes before
applying, because GRANTs persist and a reused database would otherwise let
the suite pass on leftover state without the SQL under test granting
anything — and it asserts the grants are load-bearing by stripping them and
watching anon break.
…ct the query-term doc
Two construction-time guards, and one factually wrong comment.
select('*') collision: a column renamed `created_at -> createdAt` alongside a
distinct plaintext column literally named `createdAt` makes expandAllColumns
emit the token `createdAt` twice, so addJsonbCastsV3 aliases both to
`createdAt:created_at::jsonb`. PostgREST returns the encrypted column under
that key and the plaintext one is never selected. EncryptedTable.build()'s
duplicate check only fires when two BUILDERS share a getName(), so nothing
caught it. Refuse to construct the builder.
Duplicate declared rename: two properties on the same DB column each verify
fine, then collide in mergeDeclaredTables and throw from EncryptedTable.build()
naming neither the properties nor the schemas entry. Detect it in verify.ts,
where both are known.
The class doc claimed a narrowed encryptQuery term 'fails the CHECK with 23514
for EVERY domain'. The opposite is true: the bundle defines 39 public.*_query
domains whose CHECK requires NOT (VALUE ? 'c'). The real reason full envelopes
are used is that PostgREST cannot cast a filter value, so those overloads are
unreachable and an uncast literal is ambiguous (42725) — and protect-ffi 0.28
cannot produce a v3 scalar query term at all. Also records that encrypted
like/ilike is known-broken for real substrings, with the same include_original
cause as v3 Drizzle's contains.
…anon
No supabase v3 code had ever reached a server. Every suite drove
`createMockSupabase`, an argument recorder with no SQL behind it: it can
prove the adapter EMITS `email.cs."{…}"`, never that PostgREST parses it,
that `cs` maps to `@>` on a domain column, that a full storage envelope
clears the `public.*` domain CHECK as a filter operand, or that `anon` has
the grants those operators need.
`supabase-v3-pgrest-live.test.ts` puts the real adapter, real
`@supabase/postgrest-js`, real PostgREST, real domains and the real
`eql_v3` operators in the path. Only ZeroKMS is stubbed: the domain CHECKs
are STRUCTURAL (`v`/`i`/`c` plus the domain's index terms, `v = '3'`), so
`helpers/v3-envelope.ts` builds valid envelopes directly and the suite runs
on the DATABASE_URL gate with no credentials. Cryptographic round-tripping
and ORE ordering stay `drizzle-v3/operators-live-pg`'s job — synthetic ORE
terms compare equal to themselves and carry no order semantics, so this
suite must never assert ordering.
It runs as `anon`, not as the owner: PostgREST connects as `authenticator`,
a non-superuser member of `anon`, so the Supabase grants are actually
exercised rather than bypassed by ambient superuser rights. Verified to have
teeth — emitting narrowed `encryptQuery` operands turns 7 tests red,
dropping the `like`→`cs` rewrite turns 3 red, and reverting the
`eql_v3_internal` grants turns 7 red with `permission denied for schema
eql_v3_internal` at the HTTP layer (401).
Two things this found that no unit test could:
`reloadSchemaCache` must poll the ROOT OpenAPI, not a GET against the new
table. PostgREST 12 self-heals a cache miss by scheduling a reload and
retrying that one request, so a probe GET starts succeeding while the reload
is still in flight — cold, the probe returned 200, the very next INSERT
returned 404, and a later GET returned 200 again. Root output is rendered
from the loaded cache, so a table appearing there means every verb sees it.
A pattern of exactly 3 characters is a degenerate `include_original` case:
its whole-value token IS one of the stored value's trigrams, so
`like('email','ada')` DOES match. The class doc's "a substring pattern only
matches when it equals the stored value" holds only for patterns longer than
the tokenizer window. Both behaviours are now pinned.
PostgREST starts as a CI STEP rather than a `services:` entry: service
containers start before checkout, and `authenticator` does not exist in the
postgres image until `local/postgrest-roles.sql` has run. `PGRST_URL` joins
the live-coverage guard so a missing service cannot silently skip the suite
while CI stays green.
An audit of the EQL v3 Drizzle adapter found the index/query-type wiring
correct, but several tests unable to detect the bug they guard. Three stayed
green under the exact regression they existed to catch, and two source paths
failed open.
Source:
- `contains` now rejects a needle shorter than the tokenizer's `token_length`.
A short needle builds an EMPTY bloom filter and `stored_bf @> '{}'` holds for
every row. Measured live: needles "ad", "a" and "x" each returned 3/3 seeded
rows -- "x" occurs in none of them. The floor lives in `schema/match-defaults`
so v2 and v3 share it; they build byte-identical blooms.
- `v3FromDriver` throws the new `EqlV3CodecError` rather than surfacing a raw
SyntaxError on malformed JSON or passing a wrong-shape payload through:
`v3FromDriver('5')` returned `5` typed as `Encrypted`. The shape guard accepts
SteVec documents, whose ciphertext is at `sv[0].c` with no top-level `c`.
- Removed the unreachable `sql`false`` fallback in `inArray`/`notInArray`; the
empty-list guard throws first.
Tests (each verified RED against the bug it guards, then reverted):
- `and` was indistinguishable from `or`: both predicates were true for ROW_B
alone, so swapping the operator still passed. Now paired over disjoint rows
and asserted from both directions.
- bigint filters ran against a one-row table, where `gt` returning every row
passed. Seeded a negative row they must exclude.
- `between` was only ever called with identical bounds against a constant
encrypt stub, so a min/max transposition in `range` was invisible. Added an
echoing stub that pins operand order.
- `contains` had no needle that must NOT match, and none longer than
`token_length`. The unit test searched `TEXT_S[0]` -- the empty string --
i.e. it asserted the fail-open. Now drives four needles through a substring
oracle, including a negative one.
- Seeded a third matrix row. With two, every predicate could only return [A],
[B], [A,B] or [] -- ordering ties were untestable and `eq` over-matching
undetectable. Row `i` takes `samples[min(i, len-1)]`, the scheme
`v3-matrix/matrix-live-pg` already uses.
Lock context:
The suite supplied the same context on seed and query in every test, so a
regression dropping it from the index term would drop it identically on both
sides and stay green. Added the negatives obtainable with one JWT: an
identity-bound row must not match an `eq` issued with no lock context, and must
not decrypt without it. A cross-identity proof needs a second JWT with a
different `sub` and remains a follow-up.
`lock-context.test.ts` wrapped `decryptModel` in try/catch, but it reports
denial as a `Result` and never throws -- the test ran zero assertions and would
also have passed had decryption wrongly succeeded.
Smaller pins: sql-dialect asserts operands BIND rather than interpolate;
schema-extraction uses toStrictEqual; the barrel is pinned exhaustively; codec
fixtures are realistic envelopes; the column's customType mappers are exercised.
Not included, per plan: the SQL `bloom_filter` empty-array change (contradicts
a documented semantic; needs sign-off) and the Supabase needle guard (lives on
another branch).
Three branches the PR review flagged as untested, each verified to kill its
mutant before landing:
- `select('*')` guards on `allColumns === null || .length === 0`, but only the
null arm had a caller. Pins the empty-array arm so a future `?? []` cannot
turn an unusable `*` into a silent zero-column select.
- `mergeDeclaredTables`' `if (synthesized)` false arm. Unreachable through the
factory (verifyDeclaredSchemas throws first) but the function is exported.
- `introspect()`'s happy path: both queries issued, the registry pushed as the
query parameter, and `client.end()` called — on the success path and when a
query throws. A leaked connection is invisible to every other assertion.
Two further proposed tests were dropped rather than landed: the
`addJsonbCastsV3` empty-token guard is a fast-path that falls through to the
same `return col`, so no test can kill it, and its leading-whitespace capture is
already pinned by an existing test.
…face
EQL v3 free-text search is token containment over a bloom filter, not SQL
wildcard matching: `%` is tokenized like any other character, so a `like`
pattern is a category error. The v3 Drizzle integration already omits
`like`/`ilike` in favour of `contains`; the supabase adapter now matches.
This is a surface rename, not a re-encoding. The bundle declares
CREATE OPERATOR @> (FUNCTION = eql_v3.contains,
LEFTARG = public.text_search, RIGHTARG = jsonb)
so PostgREST's `cs` already resolved through `@>` into `eql_v3.contains` →
`match_term(a) @> match_term(b)` → smallint[] bloom containment. It was never
the built-in `jsonb @> jsonb`. The emitted operator and the full-envelope
operand are unchanged; no ciphertext moves. The live PostgREST suite proves the
resolution: a 3-character needle matches while a 7-character one does not, and
neither shares the stored `c`.
`contains` is narrowed at compile time to freeTextSearch-capable domains
(text_match, text_search). The shared builder interface is now parameterised by
its own return type, so `like` is absent at every chain depth rather than only
on the first call; the runtime still throws for untyped JS callers, and passes
`like` through on a genuine plaintext column.
Three silent-failure bugs fixed alongside.
order() on ANY encrypted v3 column now throws, including the ORE-capable ones —
the non-obvious half. The `*_ord` domains are `DOMAIN … AS jsonb` and the bundle
declares no btree operator class on any domain (it lints against one), so
`ORDER BY col` resolved through jsonb's default `jsonb_cmp` and sorted by the
envelope's byte structure, first key `bf`. Stable, plausible-looking, and
meaningless, with no error. Confirmed against the live database: integer_ord has
zero operator classes and a jsonb base type. Correct ordering is
`ORDER BY eql_v3.ord_term(col)`, which PostgREST's `order=` cannot express. The
old guard rejected exactly the columns where the wrongness was obvious and
admitted the ones where it was silent. gte/lte remain correct: the comparison
operators ARE declared on the ord domains; only sorting resolves through the
missing operator class.
or() now understands PostgREST's `column.not.<op>.<value>` negation. The parser
split on the first two dots, read `not` as the operator, and so
`or('nickname.not.in.(ada,grace)')` on an encrypted column encrypted the literal
string `in.(ada,grace)` as one plaintext — a filter that silently matched
nothing. A malformed `col.not.<value>` is passed through for PostgREST to
reject rather than dropped, which would quietly widen the result set.
Raw filter() derives its query type from the operator on v3 instead of always
encrypting an equality term, so `.filter('bio','cs',…)` on a text_match column
(equality: false, freeTextSearch: true) answers the query it was given rather
than being rejected. Unknown operators throw. The v2 base still returns
'equality' verbatim: there `queryType` selects the encryptQuery narrowing, so
correcting it moves v2 ciphertext. Tracked separately.
Every new test was mutation-checked against the implementation it protects.
The include_original substring defect is untouched and still pinned by a test
asserting the broken behaviour: the operand is a storage envelope whose bloom
carries the whole needle as an extra token, so a substring only matches when it
equals the stored value or is exactly the tokenizer's 3-character window. Shared
with v3 Drizzle's contains and tracked upstream in EQL.
BREAKING CHANGE: `like`/`ilike` are removed from the EQL v3 supabase builder —
use `contains`. `order()` on an encrypted v3 column now throws; order by a
plaintext column, expose `eql_v3.ord_term()` as a generated column or view, or
use the EQL v3 Drizzle integration. v2 (`encryptedSupabase`) is unchanged.
Review of #607 found the needle guard it added was itself incomplete, and two claims it made were false. Three agents verified each finding against live Postgres before anything changed here. The guard counted UTF-16 code units; the ngram tokenizer counts CODEPOINTS. A needle of 1-2 astral-plane characters cleared the floor, produced no trigram, and so built an EMPTY bloom filter -- `stored_bf @> '{}'` matches every row. Measured live against three seeded rows: "👍👍" utf16=4 cp=2 -> 3/3 rows <- passed the guard "👍👍👍" utf16=6 cp=3 -> 0/3 rows "joh" utf16=3 cp=3 -> 1/3 rows Codepoints, not graphemes: NFD "e+combining-acute" twice is 4 codepoints but 2 grapheme clusters, and does build a non-empty filter (0/3 rows live). A grapheme floor would wrongly reject it; a byte floor would wrongly accept the emoji. The three measurements are consistent only with codepoint counting. Also rejects the empty needle under ANY tokenizer. `matchNeedleMinLength` returns undefined for the `standard` tokenizer, so the guard short-circuited and `contains(col, '')` tokenized to nothing and matched every row. Unreachable via the v3 `types` namespace today, which hardcodes ngram, but the schema permits it. Tests, written first and watched fail: - `__tests__/match-needle-guard.test.ts` pins the unit. The astral and empty-needle cases failed before the fix; the NFD case passed before and after, proving the fix does not over-correct into rejecting usable needles. Unicode is built from explicit escapes -- the precomposed NFC form is a different, 2-codepoint string, so a bare literal would silently test the opposite case depending on the file's on-disk normalisation. - The live `contains` rejection now asserts the guard's own message, not just `EncryptionOperatorError`. `operandFailure` throws the same class, and that is not hypothetical: `text_search` carries an `ore` index and cannot encrypt any non-ASCII operand ("Can only order strings that are pure ASCII"), so the astral case was passing there for the wrong reason. That constraint now has its own test, and the codepoint complement runs only on match-only columns. Two false claims corrected rather than papered over: - The changeset and the doc comment both said the floor was "shared with the v2 adapter". It is not -- `matchNeedleError` has exactly one caller, in v3. v2's `like`/`ilike` reduces to `eql_v2.bloom_filter(a) @> eql_v2.bloom_filter(b)` (cipherstash-encrypt.sql), native array containment, so v2 has the same fail-open and is NOT fixed here. Wiring it in needs its own measurement first: v2 needles carry SQL wildcards, so the floor may belong on the unwrapped term. - `decryptDenied` returned a bare boolean and treated any throw as denial, so a ZeroKMS outage read as "the identity boundary held". It now returns the message and the call sites assert /^Failed to retrieve key/, matching the assertion this PR already made in `lock-context.test.ts`. The `as SQL` cast in `inArrayOp` was reviewed and kept: drizzle returns undefined only for an empty filtered list, and the empty-list guard throws first. Restoring the old `?? sql`false`` fallback would be worse -- it would mask a `notInArray` that silently matches everything.
v3FromDriver flattened the JSON.parse failure into a message string,
discarding the SyntaxError and its stack. Widen EqlV3CodecError to accept
ErrorOptions and forward { cause }, so a caller debugging a corrupt column
keeps the original error. The package targets ES2022, so Error.cause is
available; assertEnvelope's call sites have no upstream error and keep
working since options is optional.
Also fix a misleading it.each title in sql-dialect.test.ts: vitest consumed
the `$1` as a tuple index and interpolated the builder function, rendering
"binds a hostile operand as [Function]". Reword to name positional binding
directly.
feat(stack): encryptedSupabaseV3 — EQL v3 dialect of the Supabase adapter
Follow-up to #588. Two of these are broken features, not polish; both were invisible to the existing tests because `createMockSupabase` records the arguments handed to each builder method and never builds a URL. `.in()` on an encrypted column produced a request PostgREST rejects. Every encrypted operand is a serialized envelope, dense with `"` and `,`, and postgrest-js wraps a comma-bearing element as `"…"` without escaping the quotes already inside it: in.("{"v":1,"c":"…"}",…) ^ PostgREST ends the value here -> PGRST100 Encrypted lists now go through `filter(col, 'in', …)` with each element quoted and escaped, as the `.or()` path already did. This affects v2 as well as v3 — v2's `("a@b.com")` composite literal is itself quote-bearing and was equally broken — so three existing tests that asserted the old encoding are updated. `.not(col, 'in', […])` encrypted the whole list as a single ciphertext, so the filter silently matched nothing, and emitted an unparenthesized `not.in.a,b`. The not-collector now encrypts element-wise, mirroring the regular `in` and `or(… .in. …)` paths. A PostgREST list literal on an encrypted column throws rather than silently matching nothing. `is(col, null)` is widened to every row key. `is` is never encrypted and a NULL plaintext is stored as a SQL NULL, so on a storage-only column `IS NULL` is not merely legal but the only predicate available. `is(col, true)` stays a compile error on encrypted columns. `contains()` accepts native operands on plaintext columns, which fall through to PostgREST's native containment. Relatedly, `.or([{ op: 'contains' }])` now emits `cs` for plaintext conditions too — previously only encrypted ones were translated, so plaintext containment reached the wire as `.contains.` and failed to parse — and `splitOrString` tracks `{}` so an array/jsonb literal's commas no longer split the condition mid-value. `DOMAIN_REGISTRY` is derived from `types` rather than hand-listed, so the two cannot drift. The keys are an external contract (the information_schema query parameter), and because they are now derived from `getEqlType()`, no test that also derives them can detect a corrupted domain constant — the test file pins them against a hand-written literal list for exactly that reason. Tests: a `postgrest-wire` harness runs the real PostgrestClient against a capturing fetch, so operand serialization is asserted without a database. Adds biting coverage for the three null-prototype maps (`mergeDeclaredTables`, `v3Columns` pattern-filter lookup, mutation-model rebuild) and for the unreachable scalar-queryType backstop.
An `.or()` string is only rebuilt from its parse when it references an
encrypted column — otherwise the caller's string is forwarded verbatim —
so each of these corrupts precisely the mixed encrypted/plaintext case.
Quotes were tracked only at brace depth 0, so a `}` inside a quoted array
element or jsonb string value closed the literal early and the next `"`
re-opened quoting: the following top-level comma never split and its
condition was absorbed into the operand. Quotes are now opaque at every
depth.
A stray `}` or `)` drove the depth counter negative, after which no comma
split again. Neither is a PostgREST reserved character, so `a}b` is a
valid unquoted operand. Depth now floors at zero.
`in`-list elements were split on every comma, ignoring quotes, and the
quotes were left embedded in the fragments. On an encrypted column each
fragment became its own term, so `in.("Doe, John",Smith)` never matched.
Elements are now split on top-level commas and unquoted — the inverse of
what `rebuildOrString` emits.
A parenthesized operand was read as a list for every operator, so
`eq.(foo)` encrypted a JS array rather than the string. Only `in` and the
range operators take a paren-delimited operand.
A string operand spelling `null`/`true`/`false` is now quoted: PostgREST
reads a bare `null` as SQL NULL.
Finally, `contains(col, …)` on a union key spanning an encrypted and a
plaintext column accepted an array or object. A union is only as
permissive as its strictest member; any declared column in it pins the
operand to `string`. A literal column argument was never affected.
Address review findings on the v3 Drizzle adapter.
assertEnvelope accepted `sv: []`, which satisfies `sv !== undefined` but has
no `sv[0].c` — the guard's own premise. An empty, non-array, or null `sv`
carries a ciphertext key but no ciphertext, and reached decrypt as garbage.
Require a non-empty array. No encrypt path produces this, so it is hardening
against a corrupt or hostile row rather than a live fail-open.
needleFor selected samples with `sample.length >= 3`, counting UTF-16 code
units where the production guard counts codepoints. An astral sample ('👍👍':
4 units, 2 codepoints) would be picked here and then rejected by the guard,
blaming the wrong line. Latent only because every match-domain sample is
currently ASCII. Extract it to v3-matrix/needle-for.ts and select with
matchNeedleError itself, so the two cannot diverge; it now also honours a
domain's own token_length.
Pairing `and` over disjoint rows proved it is not `or`, but left every live
`and` assertion expecting []. A constant-false `and` would satisfy that, and
the `or` tests exercise a different operator. Restore the intersecting pair
alongside it, so one live test proves `and` returns its row.
The ['email'] lock-context test pinned /^Failed to retrieve key/. Naming a
claim does not assert the token carries it: resolveLockContext is a
passthrough and ZeroKMS resolves the value server-side, so a token with no
`email` claim may be refused by authorization before key derivation is
attempted. Accept either denial class, reject infrastructure faults
explicitly. Kept loose because CI never runs this path (USER_JWT unset, #530).
Collapse exports.test.ts from three tests to two: the two typeof sweeps were
one sweep artificially split. The remaining sweep iterates Object.entries, so
it cannot silently skip a name the key list forgot.
The v2 like/ilike path has the same bloom-containment fail-open and remains
unfixed here, as the changeset already notes.
decryptOutcome counts a throw as denial, so an unseeded row makes `row.value` raise a TypeError that reads as "denied". The message assertions already reject that string, so the tests fail either way — but they fail blaming the denial regex rather than the fixture. Assert the row exists at all three `[row]` sites, not just the two decrypt-denial tests: the symmetric decrypt test has the same destructure, and leaving it unguarded is inconsistent. Raised by CodeRabbit on #607, against the older `decryptDenied` helper that returned a bare boolean. Under that version the false pass was real; the message assertions have since closed it. This is diagnostic clarity, not a correctness fix.
An assertion of absence proves nothing unless something in the same test shows the row was reachable to begin with. A sibling `it` does not couple them: vitest runs on past a failure, so the control can go red while the negative test stays green in the same run. beforeAll already throws on a failed seed, so an unseeded fixture is not the live risk. The risk it cannot catch is an operator that silently matches nothing — a constant-false predicate, or encryption no-op'ing — which produces exactly the empty result each negative test asserts. - operators-live-pg: fold the intersecting `and` pair into the disjoint block, so [] cannot pass via a constant-false `and`. Add a present-needle control to the `contains` no-match block, whose only proof that contains can match lived in a sibling test. - operators-lock-context-live-pg: assert the same eq WITH the lock context finds ROW_A before asserting it is unreachable without one. - matrix-live-pg: two-sample domains (date, timestamp) have an empty strict interior, so `gt`/`lt` were unproven there. Add one-sided bounds, which are non-empty for every domain. - operators-null-live-pg: guard the [row] destructure so a missing fixture reads as a missing fixture, not a null-handling bug. Found by audit while triaging CodeRabbit's decrypt-denial comment on #607; same defect class, sites it did not reach.
fix(stack): close v3 drizzle fail-open paths, fix false-negative tests
Fixes every finding from the PR #609 review that survived verification against HEAD. Each was reproduced with a failing test before the fix. `.or()` dropped a condition after an unbalanced opening brace or paren. The depth floor added in 8cd485d recovers from a stray CLOSING brace, but a stray OPENING one strands the counter above zero and no later comma ever splits, swallowing every condition behind it. With a plaintext column first, `referencesEncrypted` then reads false over the surviving conditions and the group is forwarded verbatim — running the swallowed condition against a ciphertext column with a plaintext operand. Braces are now quoted on emit (PostgREST treats them as structure inside `or=(…)`, so an unquoted scalar brace was malformed on the wire regardless), and `splitTopLevel` re-splits honouring quotes alone when its depth tracking does not balance. Containment literals keep the narrow reserved set, so `tags.cs.{vip}` stays unquoted. Direct `contains()` / `not(col, 'contains', …)` did not serialize their operand. postgrest-js joins array elements on `,` without quoting, so `contains('tags', ['with,comma'])` reached Postgres as two elements; its `not()` interpolates with `String(value)`, emitting `not.contains.with,comma` — no braces, wrong operator token — and `[object Object]` for a jsonb operand. Both now build the same containment literal the `.or()` path builds, and emit `cs`. `is(col, true)` compiled on queryable encrypted columns. The boolean overload was gated on the filterable keys, which exclude only the storage-only columns, so a `text_search`/`text_eq`/`*_ord` column matched it and emitted `IS TRUE` against a jsonb ciphertext. It now takes a dedicated `BK` parameter, kept distinct from the orderable `OK` so the two capability axes can diverge. The type test asserted the bug; it now asserts the compile error. `contains()` admitted arrays on plaintext SCALAR columns, emitting `@>` on `text` (42883). The plaintext operand now follows the column's own shape, mapping scalars to `never`. It distributes over `Row[K]`, which is sound only because the tuple guard already excludes every encrypted member; the residual case is documented. In-list operands spent one ZeroKMS crossing per element. Terms are now grouped by column — mandatory, since `bulkEncrypt` carries one `{table, column}` for the whole payload — and each group takes a single call, scattered back onto its original indices. Mirrors the Drizzle v3 path, keeps a per-term fallback, and rejects a length mismatch rather than silently truncating the predicate. `types` is now `satisfies Record<string, V3ColumnFactory>`. The derived `DOMAIN_REGISTRY` calls every value at module load, so a non-factory export would throw and take the supabase introspect/schema-build/verify path with it; that is now a compile error at the offending line. The blind `Object.values(types) as V3ColumnFactory[]` cast that would have silenced it is gone. In-list term collection is unified behind `collectInListTerms` — the three copies had already drifted once, leaving the `not` path unfixed.
`splitTopLevel` treated every unquoted brace and paren as structure. Neither
`{`/`(` mid-operand nor `}`/`)` is a PostgREST reserved character, so `a{b` and
`a}b` are valid unquoted scalars — and counting them desynchronised the split.
A stray closer drove the depth negative, a stray opener stranded it above zero,
and either way no later comma split: every remaining condition was absorbed
into the preceding operand.
Flooring the depth at zero fixed the closer. The opener was handled by
discarding the whole depth pass whenever the count ended unbalanced and
re-splitting on quotes alone — which trades the bug for its mirror image. Given
both a stray opener and a real containment literal, the re-split lands inside
the literal:
note.eq.a{b,tags.cs.{vip,admin}
-> ["note.eq.a{b", "tags.cs.{vip", "admin}"]
`admin}` carries no dot, so `parseOrString` drops it.
A brace or paren is structure only where the grammar can put one: opening a
logic group (`and(`, `or(`, and their `not.` forms), opening an operand right
after the operator dot, or nested inside a literal already open. Count those and
nothing else. The unbalanced-depth re-split stays as a backstop for the one case
the rule still misreads — a scalar whose brace follows an in-value dot,
`x.eq.a.{b` — where it is a recovery rather than the primary mechanism.
Both paths are reachable only through a raw-string `.or()`: `rebuildOrString`
quotes braces in scalars, so the adapter never emits the trigger itself.
Also pins the two containment operands `arrayLiteralElement` special-cases but
no wire test drove: an empty element must emit `{""}`, not the empty array `{}`,
and an element spelling `null` must emit `{"null"}`, not a SQL NULL.
…-follow-up fix(stack): escape encrypted in-list operands; widen is/contains
Reconciles the parallel EQL v3 evolution on both sides. The branch side
carried the newer stack surface (bigint domain family, freeTextSearch
tuner removal, DOMAIN_REGISTRY, supabase adapter with introspection and
or()-string hardening, null-prototype column maps, grants.ts extraction);
the main side carried the newer CLI (registry-driven help + manifest,
decoupled eql install / offerStashConfig, one-shot --database-url,
eql upgrade/status group), auth 0.41 Result API + authStrategy rename,
generation-aware isInstalled, and the eql-3.0.0-alpha.2 vendored bundle.
Resolution notes:
- stack src/tests: branch side kept; authStrategy rename and its wasm
latch taken from main; Plaintext keeps the branch bigint arm.
- encrypt-lock-context-guards now builds one client per wire format
(main's mixed-schema guard rejects a shared client).
- EncryptionV3 pins eqlVersion 3 (branch semantics); the main-side
override test now asserts the pin.
- supabase-v3.test.ts (main) removed: superseded by the branch's
supabase-v3-pgrest-live/matrix/grants/introspect suites on the
contains() surface.
- query-builder.ts: dropped main's apply-time filterColumnName calls in
favour of the branch's translate-once-at-toDbSpace boundary; removed
main's orphaned {select,keyToDb} addJsonbCastsV3 twin.
- SQL bundles/fixture: main's alpha.2 copies for now — re-vendored to
eql-3.0.0 in the follow-up FFI 0.29 upgrade.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…0.29 protect-ffi 0.28 → 0.29 and @cipherstash/eql 3.0.0-alpha.3 → 3.0.0: - Rename every EQL v3 column-domain reference to the GA naming convention: flat, prefixed domains in public (public.eql_v3_text_search, public.eql_v3_integer_ord, …). Query domains stay eql_v3.query_<name>. DOMAIN_REGISTRY keys, introspection params, drizzle DDL emission, tests, skills, and the supabase reference doc all follow. - v3 scalar/selector encryptQuery no longer throws: protect-ffi 0.29 mints term-only eql_v3.query_<name> operands, the query_jsonb needle, and bare selector hashes. EQL_V3_QUERY_UNSUPPORTED is gone; docs updated. The supabase adapter and drizzle operators keep the full-envelope operand design (PostgREST cannot cast filter values; the SQL function per proof selects the term). - Pin ste_vec mode: 'standard' in the v2 searchableJson() builders (stack and @cipherstash/schema). 0.29 flipped the library default to compat (EQL v3's op term); unpinned, every v2 JSON containment query matches nothing and new rows stop being comparable with existing ones. The v2 wire format stays byte-stable. - CLI: vendor cipherstash-encrypt-v3.sql from the pinned @cipherstash/eql package (sha256-verified against its releaseManifest) instead of a hand-vendored stack fixture. Drop the v3 Supabase bundle variant: since eql-3.0.0 one artifact installs everywhere (operator classes self-skip on insufficient_privilege; ORE-backed domains are disabled when the opclass is absent, CIP-3468). --supabase now only adds the role grants for v3. - encrypt-lock-context guards split into one client per wire format; EncryptionV3 override test asserts the v3 pin. - Skills: add the EQL v3 typed-schema section to stash-encryption, the eql/v3/drizzle integration to stash-drizzle, and re-sync stash-supabase + docs/reference/supabase-sdk.md against the current adapter (contains() instead of like/ilike, no freeTextSearch tuner, introspecting factory, single install artifact). - AGENTS.md subpath-export list gains eql/v3, eql/v3/drizzle, and v3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rms) CI against the GA bundle exposed the eql-3.0.0 term-flavour split this branch had not absorbed: plain `_ord` domains (and `text_search`) are CLLW-OPE-backed — their CHECKs require the `op` term — while only the `_ord_ore` domains keep block-ORE (`ob`). The stack emitted `ore` for every ordering domain, so every `_ord`/`text_search` INSERT failed the domain CHECK with 23514. - `indexesForCapabilities` now takes the ordering flavour, pinned by the domain name (`_ord_ore` → `ore`, everything else → `ope`); the zod schema gains the `ope` index and `FfiIndexTypeName` gains 'ope' (protect-ffi 0.29's OpeIndexOpts). - `resolveIndexType`: `orderAndRange` swaps to the ordering index the column actually carries, and equality-via-ordering resolves to `ope` or `ore` (was hardwired to `ore`). - drizzle v3 operators accept `ope` wherever `ore` satisfied the order/range and equality gates. - Tests re-pinned: catalog ordering indexes per domain, structural envelope helper emits `op` (hex bytea) for OPE domains, live-suite dispatchers treat either flavour as ordering, and the v3 text_search vs v2 parity test now asserts byte-identity modulo the ordering index (v2's wire stays block-ORE by design). The stale alpha-era `v: 2` envelope pins in schema-v3-client are corrected to `v: 3`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI's typecheck flagged `indexes.ope` access through BuildableQueryColumn: the v2 EncryptedColumn's explicit indexesValue annotation is one arm of that union and lacked the (v3-only, never set by v2) `ope` key. Also correct the two ordering-term references in the supabase reference doc (`op` for integer_ord / text_search under eql-3.0.0, not `ob`). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…emantics Round-3 CI (live, with credentials) caught the remaining eql-3.0.0 ordering-flavour fallout: - drizzle asc/desc now emit `eql_v3.ord_term_ore(col)` for the block-ORE `_ord_ore` domains — GA splits the extractor by flavour, and the bare `ord_term` call is ambiguous (42725) on ob-carrying columns. - schema-v3-client pins the `op` term (not `ob`) on text_search and integer_ord envelopes and query terms; storage-only/match columns assert BOTH ordering terms absent. - The pgrest-live envelope builder passes the renamed `ope` term flag (it silently dropped the ordering term after the COLUMN_TERMS rename, failing every seed insert with 23514). - Empty-string semantics split by flavour in matrix-live-pg: the ob ARRAY is empty for '' so text_ord_ore still rejects it, while the OPE-backed text_ord/text_search emit a scalar `op` and now accept it (new acceptance proof). - The non-ASCII-needle rejection pin applies only to the ore-flavoured match domain — OPE terms encrypt non-ASCII fine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
EQL v3 typed schema + strongly-typed client (
@cipherstash/stack/eql/v3,@cipherstash/stack/v3)Builds the full EQL v3 authoring and querying surface on the eql-3.0.0 GA release and protect-ffi 0.29: a
typesnamespace with one member per modelled EQL v3 SQL domain, a strongly-typed v3 client (@cipherstash/stack/v3) that derives input/output types from the schema and rejects misuse at compile time, an EQL v3 Supabase adapter with database introspection, an EQL v3 Drizzle integration, and the CLI install path (stash eql install --eql-version 3). v2 is unchanged; pick the model by import path.Usage
Mix any v3 domains in one table — each column declares its own type and query capabilities. The
types.*member name maps 1:1 to the flatpublic.eql_v3_<name>domain (strip theeql_v3_prefix, PascalCase each segment):Plaintext types are inferred per domain;
Dateandbigintvalues work directly (bigintis marshalled losslessly by protect-ffi 0.28+, with i64 bounds enforced before the FFI):Queryability is enforced at compile time — storage-only columns can't be queried.
typesmembers & capabilitiesOne member per modelled EQL v3 domain. Each maps to an EQL v3 SQL type and exposes
getQueryCapabilities()/isQueryable(). Every domain is fully described by its type — there is no capability-bearing or tuning chain (the former.freeTextSearch(opts)tuner is gone; match indexes use the default configuration).types.Integerpublic.eql_v3_integerEqtypes.TextEqpublic.eql_v3_text_eqOrd,OrdOretypes.IntegerOrdpublic.eql_v3_integer_ordTextMatchtypes.TextMatchpublic.eql_v3_text_matchTextSearchtypes.TextSearchpublic.eql_v3_text_searchCovered families (40 domains):
integer/smallint/bigint,real/double,numeric,date,timestamp,text*,boolean→ inferred asnumber/bigint/Date/string/boolean. TheDOMAIN_REGISTRYderives itself from these factories and keys Supabase introspection.Not yet modelled (deliberate follow-up): the
*_ord_opetwins andtext_search_ore— they exist in the eql-3.0.0 SQL and protect-ffi 0.29 supports theopeindex, but the SDK surface, matrix coverage, and introspection classification for them are a separate increment. Introspection reports them as unmodelled EQL columns and guards queries against them.Strongly-typed client (
@cipherstash/stack/v3)EncryptionV3mirrorsEncryption;typedClientretypes an existing client. Both re-export the v3typesnamespace and table API.encrypt/encryptQuerypin the plaintext to the column's domain type and constrainqueryTypeto the column's capabilities at compile time.encryptModel/bulkEncryptModelsvalidate schema-column fields against their inferred plaintext type; passthrough fields are untouched.decryptModel/bulkDecryptModelsreturn the precise plaintext model, reconstructingDatevalues (date and timestamp casts) via per-table reconstructors precomputed at construction.EncryptionV3pinseqlVersion: 3(a v2-mode client cannot resolve v3 concrete-type columns);Encryptionauto-detects v3 tables and rejects mixed v2 + v3 schema sets — one client emits exactly one wire format.Supabase adapter (
encryptedSupabaseV3)An EQL v3 dialect of the Supabase integration: introspects the database over the optional
pgpeer (EQL v3 domains classified viaDOMAIN_REGISTRY), synthesizes or verifies declared schemas, expandsselect('*')from the introspected column list, aliases renamed columns back to their JS property names with::jsonbcasts, encrypts filter operands (full storage envelopes — PostgREST cannot cast a filter value to theeql_v3.query_<name>twins), translatescontains()to PostgRESTcsbloom containment (like/ilikeare rejected on encrypted columns by design), hardens.or()string parsing/rebuilding, and guards unmodelled EQL domains. Null-prototype column maps close the__proto__/constructorlookup holes.Drizzle integration (
@cipherstash/stack/eql/v3/drizzle)makeEqlV3Column(drizzlecustomTypeemitting the domain DDL),extractEncryptionSchemaV3, andcreateEncryptionOperatorsV3(eq/gt/gte/lt/lte/between/inArray/notInArray/contains — nolike/ilike; operands are full envelopes compared via the two-argeql_v3.*functions).CLI
stash eql install --eql-version 3installs the eql-3.0.0 bundle, vendored from the pinned@cipherstash/eql@3.0.0npm package and sha256-verified against its release manifest. Since 3.0.0 one artifact installs everywhere: operator-class statements self-skip oninsufficient_privilegeand the bundle disables ORE-backed domains when the opclass is absent (CIP-3468), so the separate v3 Supabase variant is gone.--supabasestill applies the role grants (eql_v3andeql_v3_internal— the SECURITY INVOKER extractors need both).isInstalledis generation-aware and treats a stale pre-GA install as not installed.protect-ffi 0.29 / eql-3.0.0 re-baseline
public.eql_v3_<name>. Query domains stayeql_v3.query_<name>. Databases installed from an alpha bundle must be re-installed (the bundle replaces the schema).encryptQueryno longer throws — protect-ffi 0.29 mints term-only query operands (EQL_V3_QUERY_UNSUPPORTEDis gone). The adapters keep full-envelope operands where SQL-side casting isn't available (PostgREST).compat(EQL v3'sopterm). The v2searchableJson()builders (stack +@cipherstash/schema) now pinmode: 'standard'so existing v2 encrypted JSON columns stay queryable and the v2 wire format stays byte-stable.@cipherstash/eql(release-manifest-verified) instead of a hand-vendored fixture;installEqlV3IfNeededcheckseql_v3.version()against the pinned release.Docs & skills (ship in the
stashtarball)skills/stash-encryptiongains the full EQL v3 typed-schema section;skills/stash-drizzlegains theeql/v3/drizzlesection;skills/stash-supabaseanddocs/reference/supabase-sdk.mdare re-synced against the current adapter (introspecting factory,contains(), single install artifact, GA naming).Notes
Live client/pg tests require CipherStash credentials (CI provides them; the
require-cs-secretsaction fails loudly when missing). Thesupabase-v3-grants-pg/supabase-v3-introspect-pgsuites are flaky over a PgBouncer transaction-mode pooler (session advisory locks +SET ROLE) — they are written for the CI service Postgres.