Skip to content

BigQuery: support CREATE VECTOR INDEX - #2437

Open
moshap-firebolt wants to merge 1 commit into
apache:mainfrom
moshap-firebolt:bigquery-create-vector-index
Open

BigQuery: support CREATE VECTOR INDEX#2437
moshap-firebolt wants to merge 1 commit into
apache:mainfrom
moshap-firebolt:bigquery-create-vector-index

Conversation

@moshap-firebolt

Copy link
Copy Markdown
Contributor

BigQuery can create a vector index for approximate nearest-neighbor search over
an embedding column:

CREATE [OR REPLACE] VECTOR INDEX [IF NOT EXISTS] <name>
ON <table>(<column>)
OPTIONS(index_type = 'IVF', distance_type = 'COSINE', ...)

VECTOR is not a keyword, so parse_create previously failed with
Expected: an object type after CREATE, found: VECTOR.

Changes

  • Add vector, or_replace and options fields to CreateIndex. VECTOR is
    a modifier on CREATE INDEX (like EXTERNAL on CREATE TABLE), so it reuses
    the existing node rather than a new statement variant. Display renders
    CREATE [OR REPLACE ]VECTOR INDEX ... plus a trailing OPTIONS(...).
  • Parse the form in parse_create via a small parse_create_vector_index
    helper; the OPTIONS(...) clause reuses parse_options / SqlOption, so it
    parses and renders the same way as CREATE TABLE / CREATE VIEW OPTIONS.
  • Plain CREATE INDEX is unchanged (all three fields default to false/empty).
  • New test parse_bigquery_create_vector_index in tests/sqlparser_bigquery.rs
    verifies the round-trip and covers OR REPLACE, IF NOT EXISTS, multi-part
    names and the OPTIONS-less form.

Docs: https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_vector_index_statement

BigQuery can create a vector index for approximate nearest-neighbor search over
an embedding column:

    CREATE [OR REPLACE] VECTOR INDEX [IF NOT EXISTS] <name>
    ON <table>(<column>)
    OPTIONS(index_type = 'IVF', distance_type = 'COSINE', ...)

`VECTOR` is not a keyword, so `parse_create` previously failed with
`Expected: an object type after CREATE, found: VECTOR`.

### Changes
- Add `vector`, `or_replace` and `options` fields to `CreateIndex`. `VECTOR` is
  a modifier on `CREATE INDEX` (like `EXTERNAL` on `CREATE TABLE`), so it reuses
  the existing node rather than a new statement variant. `Display` renders
  `CREATE [OR REPLACE ]VECTOR INDEX ...` plus a trailing `OPTIONS(...)`.
- Parse the form in `parse_create` via a small `parse_create_vector_index`
  helper; the `OPTIONS(...)` clause reuses `parse_options` / `SqlOption`, so it
  parses and renders the same way as `CREATE TABLE` / `CREATE VIEW` OPTIONS.
- Plain `CREATE INDEX` is unchanged (all three fields default to false/empty).
- New test `parse_bigquery_create_vector_index` in `tests/sqlparser_bigquery.rs`
  verifies the round-trip and covers `OR REPLACE`, `IF NOT EXISTS`, multi-part
  names and the OPTIONS-less form.

Docs: https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_vector_index_statement

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant