Reference for Vector and Text Search¶

On AWS¶

On GCP¶

Create Vector Table¶

Syntax:

CREATE TABLE IF NOT EXISTS keyspace.table (
  column1 type,
  column2 type,
  vector_column_name vector<float, n>,
  PRIMARY KEY (column1, column2)
);

Example:

CREATE TABLE IF NOT EXISTS myapp.comments (
  record_id timeuuid,
  id uuid,
  commenter text,
  comment text,
  comment_vector vector<float, 5>,
  created_at timestamp,
  PRIMARY KEY (id, created_at)
);

Create Vector Index¶

Syntax:

CREATE CUSTOM INDEX [ IF NOT EXISTS ] index_name
ON table_name (vector_column_name)
USING 'vector_index'
[ WITH OPTIONS = { options } ];

Available options:

Example:

CREATE CUSTOM INDEX comment_ann_index
ON myapp.comments(comment_vector)
USING 'vector_index'
WITH OPTIONS = {
  'similarity_function': 'COSINE',
  'maximum_node_connections': '16',
  'quantization': 'i8',
  'oversampling': '5.0',
  'rescoring': 'true'
};

See Global Secondary Indexes - Vector Index in the ScyllaDB documentation for details.

Drop Vector Index¶

Syntax:

DROP INDEX [ IF EXISTS ] index_name;

Example:

DROP INDEX IF EXISTS comment_ann_index;

Run Similarity Search¶

Syntax:

SELECT column1, column2, ...
FROM keyspace.table
[ WHERE condition ]
ORDER BY vector_column_name ANN OF [v1, v2, ..., vn]
LIMIT k
[ ALLOW FILTERING ];

Example:

SELECT id, commenter, comment
FROM myapp.comments
ORDER BY comment_vector ANN OF [0.12, 0.34, 0.56, 0.78, 0.91]
LIMIT 3;

See Data Manipulation - SELECT - Vector Queries in the ScyllaDB documentation for details.

To combine similarity search with metadata constraints using global or local indexes, see Filtering Vector and Text Search Results.

Retrieve Similarity Scores¶

Use one of the built-in similarity functions to return a similarity score for each row. The function must match the similarity_function configured on your vector index.

Available functions:

• similarity_cosine(<vector>, <vector>)

• similarity_dot_product(<vector>, <vector>)

• similarity_euclidean(<vector>, <vector>)

Each argument can be either a vector column name or a vector literal. Both arguments must have the same dimension.

Syntax:

SELECT column1, similarity_cosine(vector_column, [v1, ..., vn])
FROM keyspace.table
ORDER BY vector_column ANN OF [v1, ..., vn]
LIMIT k;

Example:

SELECT commenter, comment,
       similarity_cosine(comment_vector, [0.12, 0.34, 0.56, 0.78, 0.91])
       AS similarity
FROM myapp.comments
ORDER BY comment_vector ANN OF [0.12, 0.34, 0.56, 0.78, 0.91]
LIMIT 3;

Each function returns a float in the range [0, 1]. Values closer to 1 indicate greater similarity. The similarity_euclidean and similarity_dot_product functions do not perform vector normalization prior to computing similarity.

See Vector Similarity Functions in the ScyllaDB documentation for details.

Create Full-Text Index¶

Syntax:

CREATE CUSTOM INDEX [ IF NOT EXISTS ] [ index_name ]
ON keyspace.table (text_column)
USING 'fulltext_index'
[ WITH OPTIONS = { options } ];

Example:

CREATE CUSTOM INDEX articles_body_fts ON blog.articles(body)
USING 'fulltext_index'
WITH OPTIONS = {'analyzer': 'english'};

The target column must be of type text, varchar, or ascii, and cannot be a partition key column. See Which Columns Can Be Indexed.

Available options — values are case-insensitive, and neither option can be changed after the index is created:

Analyzers¶

See Choosing an Analyzer for guidance on picking one.

Drop Full-Text Index¶

Syntax:

DROP INDEX [ IF EXISTS ] index_name;

Example:

DROP INDEX IF EXISTS blog.articles_body_fts;

Run Full Text Search¶

Syntax:

SELECT column1, column2, ...
FROM keyspace.table
WHERE BM25(text_column, 'query') > 0
ORDER BY BM25(text_column, 'query')
LIMIT n;

Example:

SELECT id, body
FROM blog.articles
WHERE BM25(body, 'database AND distributed') > 0
ORDER BY BM25(body, 'database AND distributed')
LIMIT 10;

All four clauses are required, the search term must be identical in WHERE and ORDER BY, the comparison must be > 0, and LIMIT must not exceed 1000. No other WHERE restriction is permitted. See Running Full Text Search Queries for the complete set of rules.

Query Expression Syntax¶

Boolean operators are recognized only in upper case; a lowercase and/or/not is treated as a search term. Fuzzy matching (term~1) and prefix matching (term*) are not available. Stemming, stop-word removal, lowercasing, and punctuation splitting all depend on the analyzer — see Choosing an Analyzer. For the query expression rules, see Text Query Language.

Create a Table¶

resource.create_table(
    TableName="comments",
    KeySchema=[{"AttributeName": "id", "KeyType": "HASH"}],
    AttributeDefinitions=[{"AttributeName": "id", "AttributeType": "S"}],
    BillingMode="PAY_PER_REQUEST",
)

Insert an Item with a Vector¶

The vector is a list of numbers. In the DynamoDB resource interface, numbers are represented as Decimal:

from decimal import Decimal

table = resource.Table("comments")
table.put_item(
    Item={
        "id": "1",
        "comment": "I like vector search in ScyllaDB.",
        "comment_vector": [Decimal(str(x)) for x in (0.12, 0.34, 0.56, 0.78, 0.91)],
    }
)

Create a Vector Index¶

client.update_table(
    TableName="comments",
    VectorIndexUpdates=[
        {
            "Create": {
                "IndexName": "comment_ann_index",
                "VectorAttribute": {
                    "AttributeName": "comment_vector",
                    "Dimensions": 5,
                },
            }
        }
    ],
)

The index builds asynchronously. Poll describe_table until the index status is ACTIVE:

import time

while True:
    indexes = client.describe_table(TableName="comments")["Table"].get(
        "VectorIndexes", []
    )
    if indexes and indexes[0]["IndexStatus"] == "ACTIVE":
        break
    time.sleep(1)

Run a Similarity Search¶

Pass the query vector in VectorSearch and use Limit for the number of nearest neighbors (k). The index projects only the key, so read the full item for each match:

from decimal import Decimal

query_vector = [Decimal(str(x)) for x in (0.12, 0.34, 0.56, 0.78, 0.91)]
response = table.query(
    IndexName="comment_ann_index",
    VectorSearch={"QueryVector": query_vector},
    Limit=3,
)
for match in response["Items"]:
    item = table.get_item(Key={"id": match["id"]})["Item"]
    print(item["comment"])

Create a Vector and Text Search Cluster¶

Create a new cluster with Vector and Text Search enabled:

curl -X POST "https://api.cloud.scylladb.com/account/{ACCOUNT_ID}/cluster" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "clusterName": "my-vector-cluster",
    "cloudProviderId": 1,
    "regionId": 1,
    "numberOfNodes": 3,
    "instanceId": 62,
    "replicationFactor": 3,
    "vectorSearch": {
      "nodeCount": 1,
      "defaultInstanceTypeId": 176
    }
  }'

Deploy Indexing Nodes¶

Deploy indexing nodes in the specified datacenter (DC):

curl -X POST "https://api.cloud.scylladb.com/account/{ACCOUNT_ID}/cluster/{CLUSTER_ID}/dc/{DC_ID}/vector-search" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "nodeCount": 1,
    "defaultInstanceTypeId": 176
  }'

Delete Indexing Nodes¶

Delete indexing nodes from the specified datacenter (DC):

curl -X DELETE "https://api.cloud.scylladb.com/account/{ACCOUNT_ID}/cluster/{CLUSTER_ID}/dc/{DC_ID}/vector-search" \
  -H "Authorization: Bearer YOUR_API_TOKEN"

Resize Indexing Nodes¶

Resize the Vector and Text Search deployment in the specified datacenter (DC) by changing the node count or instance type:

curl -X PATCH "https://api.cloud.scylladb.com/account/{ACCOUNT_ID}/cluster/{CLUSTER_ID}/dc/{DC_ID}/vector-search" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "nodeCount": 3,
    "defaultInstanceTypeId": 177
  }'

Parameters:

• nodeCount (integer, 1–27, required) — number of indexing nodes, distributed evenly across cluster racks.

• defaultInstanceTypeId (integer, required) — ID of the instance type. See Supported Instance Types or use the deployment endpoint with target=VECTOR_SEARCH to list available types.

Get Indexing Nodes¶

Get information about indexing nodes in the specified datacenter (DC):

curl -X GET "https://api.cloud.scylladb.com/account/{ACCOUNT_ID}/cluster/{CLUSTER_ID}/dc/{DC_ID}/vector-search" \
  -H "Authorization: Bearer YOUR_API_TOKEN"

Example response:

{
  "data": {
    "availabilityZones": [
      {
        "azid": "use1-az4",
        "nodes": [
          {
            "id": 337,
            "instanceTypeId": 176,
            "status": "ACTIVE"
          }
        ]
      },
      {
        "azid": "use1-az5",
        "nodes": [
          {
            "id": 338,
            "instanceTypeId": 176,
            "status": "ACTIVE"
          }
        ]
      },
      {
        "azid": "use1-az2",
        "nodes": [
          {
            "id": 339,
            "instanceTypeId": 176,
            "status": "ACTIVE"
          }
        ]
      }
    ]
  }
}

Get Your Account ID¶

curl -X GET "https://api.cloud.scylladb.com/account/default" \
  -H "Authorization: Bearer YOUR_API_TOKEN"

Response:

{
  "error": "",
  "data": {
    "accountId": 12345,
    "name": "my-account",
    "userId": "12345"
  }
}

Get Your Cluster ID¶

curl -X GET "https://api.cloud.scylladb.com/account/{ACCOUNT_ID}/clusters" \
  -H "Authorization: Bearer YOUR_API_TOKEN"

Get Your DC ID¶

curl -X GET "https://api.cloud.scylladb.com/account/{ACCOUNT_ID}/cluster/{CLUSTER_ID}/dcs" \
  -H "Authorization: Bearer YOUR_API_TOKEN"

Get Available Indexing Node Instance Types¶

List available indexing node instance types for a specific cloud provider and region by passing target=VECTOR_SEARCH:

curl -X GET "https://api.cloud.scylladb.com/deployment/cloud-provider/{CLOUD_PROVIDER_ID}/region/{REGION_ID}?target=VECTOR_SEARCH" \
  -H "Authorization: Bearer YOUR_API_TOKEN"

The response contains a list of instance types with their IDs. Use the instance type ID as the defaultInstanceTypeId parameter when deploying or resizing indexing nodes.

Track Operation Status¶

All asynchronous Vector and Text Search operations (deploy, resize, delete) return a ClusterRequest object with a request ID. You can poll for the operation status:

curl -X GET "https://api.cloud.scylladb.com/account/{ACCOUNT_ID}/cluster/request/{REQUEST_ID}" \
  -H "Authorization: Bearer YOUR_API_TOKEN"

The status field in the response is one of: QUEUED, IN_PROGRESS, COMPLETED, or FAILED.