ScyllaDB Docs ScyllaDB Cloud Vector and Text Search Filtering Vector and Text Search Results

For AI agents: a documentation index is available at https://cloud.docs.scylladb.com/master/llms.txt. A Markdown version of this page is at https://cloud.docs.scylladb.com/master/vector-search/vector-search-filtering.md.

Filtering Vector and Text Search Results¶

Filtering lets you combine similarity search with metadata constraints, so results are both semantically relevant and meet your business requirements. This page explains how to use filtering with global and local vector indexes in ScyllaDB.

Note

Filtering is available through CQL. Combining a similarity search with metadata constraints is not available through the Alternator (DynamoDB) API in the 2026.2.x series. See Feature Availability in Alternator.

Overview¶

Typical filtering use cases include:

Multi-tenant isolation — tenant_id = 'acme'
Recency — created_at >= '2024-01-01'
Geo-filtering — region IN ('EU', 'US')
Access control — visibility = 'public'

Without filtering, you would need to retrieve a large set of similar vectors and then filter them in your application. ScyllaDB filtering pushes this work into the database, reducing network overhead and application complexity.

ScyllaDB supports two types of vector indexes for filtering:

Global vector indexes — index all vectors in a table. Filter on primary key columns, or on non-primary-key columns added as filtering columns to the index.
Local vector indexes — index vectors within a single partition. Significantly faster than global indexes because they search only a single partition’s index instead of the entire index space.

Caution

For best filtering performance, design your schema so that the columns you filter on are part of the partition key, and use a local vector index. This ensures that only equality (=) filters on partition key columns are needed, which is the fastest path. Global index filtering and inequality/IN operators are substantially slower (see details below).

Filtering with Global Vector Indexes¶

A global vector index indexes all vector data stored in a table. You can filter results using columns that are part of the table’s primary key (partition key and clustering columns).

Caution

Searching through a global index is always much slower than searching through a local index, because ScyllaDB must search the entire index space across all partitions and then post-filter the results. The more selective the filter (i.e., the fewer rows that match), the slower the query, because more index entries must be scanned to find enough matching results.

Global index queries also require the ALLOW FILTERING option in the SELECT statement.

Whenever possible, prefer local vector indexes for filtered vector search.

Example Table Schema¶

The examples in this section use the following table with a composite partition key:

CREATE TABLE IF NOT EXISTS myapp.comments_vs (
  commenter text,
  comment text,
  comment_vector VECTOR<FLOAT, 5>,
  created_at timestamp,
  discussion_board_id int,
  country text,
  lang text,
  PRIMARY KEY ((commenter, discussion_board_id), created_at)
);

Creating a Global Vector Index¶

The syntax is the same as creating a standard vector index:

CREATE CUSTOM INDEX IF NOT EXISTS global_ann_index
ON myapp.comments_vs(comment_vector)
USING 'vector_index'
WITH OPTIONS = { 'similarity_function': 'DOT_PRODUCT' };

With a global vector index, you can filter on any column that is part of the base table’s primary key. To also filter on non-primary-key columns, add them as filtering columns to the index — see Filtering on Non-Primary-Key Columns.

Querying with Filtering¶

You can filter on primary key columns (commenter, discussion_board_id, created_at) in your ANN query:

SELECT commenter, comment FROM myapp.comments_vs
WHERE created_at = '1970-01-01 00:01:04'
ORDER BY comment_vector ANN OF [0.1, 0.2, 0.3, 0.4, 0.5] LIMIT 5
ALLOW FILTERING;

The LIMIT keyword limits the number of results returned after applying the filter, not before. Internally, the ANN search first retrieves candidate vectors by similarity, then applies the filter predicates. If the filter eliminates most candidates, the final result set may be smaller than the requested LIMIT.

Local Vector Indexes¶

A local vector index creates a separate vector index per partition. This is more efficient than global filtering when you frequently query within a specific partition (e.g., per tenant, per user, per discussion board).

Caution

On ScyllaDB versions earlier than 2026.3, the columns that make up a local vector index’s partition key must match the partition key of the base table. Starting with ScyllaDB 2026.3, a local vector index can also be keyed on non-primary-key columns.

Creating a Local Vector Index¶

The local vector index syntax specifies the partition key columns in parentheses before the vector column:

CREATE CUSTOM INDEX IF NOT EXISTS local_ann_index
ON myapp.comments_vs((commenter, discussion_board_id), comment_vector)
USING 'vector_index'
WITH OPTIONS = { 'similarity_function': 'DOT_PRODUCT' };

In this example:

(commenter, discussion_board_id) — the partition key columns. These must match the base table’s partition key.
comment_vector — the vector column to index.

For each unique partition key value, ScyllaDB maintains a separate vector index, which keeps index sizes small and queries local to a single node.

Local Vector Indexes on Non-Primary-Key Partition Columns¶

Starting with ScyllaDB 2026.3, the partition key of a local vector index does not have to be part of the base table’s partition key. You can key a local vector index on any primary or non-primary-key column of the base table, which scopes each per-partition index by that column’s value.

CREATE CUSTOM INDEX IF NOT EXISTS local_ann_by_lang
ON myapp.comments_vs((lang), comment_vector)
USING 'vector_index'
WITH OPTIONS = { 'similarity_function': 'DOT_PRODUCT' };

In this example, lang is a regular (non-primary-key) column of the base table that becomes the index partition key. ScyllaDB maintains a separate vector index for each distinct lang value, and queries specify the lang value with an equality (=) predicate in the WHERE clause:

SELECT commenter, comment FROM myapp.comments_vs
WHERE lang = 'en'
ORDER BY comment_vector ANN OF [0.1, 0.2, 0.3, 0.4, 0.5] LIMIT 5;

The column used as the index partition key must be:

Of a native data type allowed for filtering columns — every native type except counter and duration.
A primary key column or a regular column. Static columns are not supported.

Note

Using a non-primary-key column as the index partition key scopes the search to that column’s value. To filter results on additional non-primary-key columns, add them as filtering columns after the vector column (this requires ALLOW FILTERING) — see Filtering on Non-Primary-Key Columns.

Querying with a Local Vector Index¶

When using a local vector index, you must specify the full partition key in the WHERE clause:

SELECT commenter, comment FROM myapp.comments_vs
WHERE commenter = 'Alice' AND discussion_board_id = 42
ORDER BY comment_vector ANN OF [0.1, 0.2, 0.3, 0.4, 0.5] LIMIT 5;

You can also combine partition key filtering with clustering column filtering. Filtering on clustering columns goes beyond the index’s partition key, so the query requires ALLOW FILTERING:

SELECT commenter, comment FROM myapp.comments_vs
WHERE commenter = 'Alice' AND discussion_board_id = 42
  AND created_at >= '2024-01-01'
ORDER BY comment_vector ANN OF [0.1, 0.2, 0.3, 0.4, 0.5] LIMIT 5
ALLOW FILTERING;

Caution

Inequality operators (>, <, >=, <=) and the IN operator are always slow in vector search queries, regardless of whether they are applied to partition key or clustering columns. These operators force ScyllaDB to search a much larger portion of the index space than an equality (=) filter. The slowdown is proportional to the filter’s selectivity - the fewer rows that match the filter, the more index entries must be scanned, and the slower the query becomes.

For best performance, design your schema so that the columns you need to filter on are partition key columns queried with equality (``=``) operators. Use a local vector index so that the search is confined to a single partition’s index.

Filtering on Non-Primary-Key Columns¶

Starting with ScyllaDB 2026.2, you can filter similarity search results on columns that are not part of the base table’s primary key. To do this, add each column you want to filter on as a filtering column after the vector column in the index definition. This applies to both global and local vector indexes.

Note

Adding non-primary-key filtering columns after the vector column is a different mechanism from keying a local vector index on a non-primary-key partition column. The latter scopes the per-partition index by a column’s value; the former lets you filter query results on additional columns.

Global vector index with a filtering column:

CREATE CUSTOM INDEX IF NOT EXISTS global_ann_filtered
ON myapp.comments_vs(comment_vector, country)
USING 'vector_index'
WITH OPTIONS = { 'similarity_function': 'DOT_PRODUCT' };

SELECT commenter, comment FROM myapp.comments_vs
WHERE country = 'US'
ORDER BY comment_vector ANN OF [0.1, 0.2, 0.3, 0.4, 0.5] LIMIT 5
ALLOW FILTERING;

Local vector index with a filtering column:

CREATE CUSTOM INDEX IF NOT EXISTS local_ann_filtered
ON myapp.comments_vs((commenter, discussion_board_id), comment_vector, country)
USING 'vector_index'
WITH OPTIONS = { 'similarity_function': 'DOT_PRODUCT' };

SELECT commenter, comment FROM myapp.comments_vs
WHERE commenter = 'Alice' AND discussion_board_id = 42 AND country = 'US'
ORDER BY comment_vector ANN OF [0.1, 0.2, 0.3, 0.4, 0.5] LIMIT 5
ALLOW FILTERING;

In both examples, country is a regular (non-primary-key) column added after the comment_vector vector column, which makes it available for filtering. Filtering on a filtering column always requires ALLOW FILTERING, with both global and local indexes. With the local index, you must additionally specify the index’s partition key columns with equality (=) predicates.

Filtering columns must be of a native data type allowed for filtering — every native type except counter and duration. Static columns are not supported.

Choosing Between Global and Local Indexes¶

Criteria	Global Vector Index	Local Vector Index
Index scope	All rows in the table	Rows within a single partition
Filter columns	Primary key and non-primary-key columns	Primary key and non-primary-key columns
Requires partition key in WHERE	No	Yes (the index’s partition key columns)
Performance at scale (>10M vectors)	Always much slower (searches entire index space)	Fast (searches only one partition’s index)
Use case	Cross-partition similarity search	Per-tenant, per-user, or scoped search
`ALLOW FILTERING` required	Yes (when using WHERE clause)	Only when the `WHERE` clause filters beyond the index’s partition key columns (e.g., on clustering or filtering columns)

General guidance:

Always prefer local indexes over global indexes for filtered vector search. Local indexes search only a single partition’s index, while global indexes must search the entire index space, making them significantly slower.
Design your schema so that columns you filter on are part of the partition key. This lets you use a local vector index with equality (=) filters on the partition key - the fastest possible filtering path.
Use global indexes only when you genuinely need to search across all data without knowing the partition key in advance. Be aware that performance degrades as the dataset grows.
Avoid inequality and IN operators in filtered vector queries. They force the database to scan a larger portion of the index, with slowdown proportional to selectivity (fewer matching rows = slower query).

If both a global and a local vector index exist on the same vector column, ScyllaDB automatically selects the local index when the partition key is specified in the query, as it provides better performance.

Limitations¶

The TOKEN function, CONTAINS operator, and DISTINCT keyword are not supported in vector queries.
On ScyllaDB versions earlier than 2026.2, filtering on columns not in the primary key is not supported. Starting with ScyllaDB 2026.2, you can filter on non-primary-key columns with both global and local vector indexes by adding them as filtering columns after the vector column. Starting with ScyllaDB 2026.3, a local vector index can also be keyed on a non-primary-key column.

What’s Next¶

Working with Vector and Text Search — vector data type, index creation, and ANN queries.
Quantization and Rescoring — reduce memory usage while maintaining search quality.
Vector and Text Search Concepts — architecture and data flow.

Was this page helpful?