Vector and Text Search Glossary¶

This glossary defines key terms related to Vector and Text Search in ScyllaDB. It covers core concepts essential to understanding how vectors and text are stored, indexed, and queried.

• ANN (Approximate Nearest Neighbor) Search — A search technique that efficiently finds data points in a large dataset that are most similar to a given query vector. Instead of looking for an exact match, ANN speeds up the search by accepting results that are close enough — making it ideal for working with large datasets and high-dimensional vector spaces in applications like semantic search, recommendations, and generative AI.

• Analyzer — The text-processing pipeline a full-text index applies to both the stored text and the query: tokenization, lowercasing, stop-word removal, and stemming. Chosen per index with the analyzer option and fixed for the life of that index. See Choosing an Analyzer.

• BM25 (Best Matching 25) — The relevance-ranking function used by ScyllaDB’s full text search. It scores a row on how often the query terms occur in it, discounted by how common those terms are across the index and by the length of the indexed text. Results of a BM25() query are returned highest score first. See Relevance Ranking.

• CDC Reader — A Change Data Capture (CDC) consumer that propagates base table mutations to the vector index. ScyllaDB uses two readers: a fine-grained reader with sub-second intervals for low-latency updates, and a wide-framed reader with a 30-second safety interval that ensures consistency.

• Embedding — A vector generated by a machine learning model to represent raw data in a numerical form. ScyllaDB can store and query embeddings generated by an external tool and inserted into the database.

• Filtering — The ability to combine an ANN similarity search with predicate constraints on primary key columns. ScyllaDB supports filtering via global vector indexes (with ALLOW FILTERING) and local vector indexes (with partition key restrictions). See Filtering.

• Full Text Search — A search technique that retrieves rows by matching the terms of a query against the text stored in a column, and ranks them by relevance with BM25. It complements vector search, which retrieves by similarity of meaning. See Working with Full Text Search.

• Full-Text Index — In ScyllaDB, an inverted index built on a text, varchar, or ascii column with CREATE CUSTOM INDEX ... USING 'fulltext_index'. It is maintained on the same indexing nodes as the vector index, populated through CDC, and configured at creation time by the analyzer and positions options. See Creating a Full-Text Index.

• Global Vector Index — A vector index that spans all partitions and enables cluster-wide similarity search. Global indexes support filtering on the base table’s primary key columns with ALLOW FILTERING. See Global Vector Indexes.

• HNSW (Hierarchical Navigable Small World) — A graph-based algorithm for Approximate Nearest Neighbor search. It organizes vectors into a multi-layered graph, allowing efficient navigation from coarse to fine resolution. ScyllaDB’s vector index uses the HNSW algorithm implemented by the USearch library.

• Hybrid Search — Combining full text search and vector search over the same data and merging the two ranked result sets, so that exact term matches and semantic similarity both contribute to relevance. In ScyllaDB a table can carry both index types, but a single query cannot use both — run the two queries and fuse the results, for example with Reciprocal Rank Fusion. See Combining Full Text and Vector Search.

• Inverted Index — The data structure behind full text search. Rather than mapping rows to their text, it maps each term to the rows that contain it, which is what makes term lookups fast without scanning the base table.

• Local Vector Index — A per-partition vector index that is co-located with the data it indexes. Local indexes require the full partition key in the WHERE clause and support efficient filtered similarity search within a single partition. See Local Vector Indexes.

• Oversampling — An index option (range 1.0-100.0) that controls how many candidate vectors the index evaluates internally before returning the top-k results. Higher oversampling improves recall at the cost of higher latency. See Quantization and Rescoring.

• Quantization — A technique that reduces index memory usage by storing vectors in lower-precision formats (e.g., f16, i8, b1) instead of the default f32. Lower precision reduces memory but may decrease accuracy; combine with rescoring to recover precision. See Quantization and Rescoring.

• Rescoring — A post-processing step where the index re-ranks candidate results using the original full-precision vectors from disk, improving accuracy after quantized search. Enabled by setting 'rescoring': 'true' in the index options. See Quantization and Rescoring.

• Semantic Search — A type of similarity search that compares the meaning of a query and data items using vector embeddings. It enables context-aware retrieval by focusing on semantic relevance rather than exact terms.

• Similarity Function (Distance Metric) — A mathematical function that measures how close two vectors are. In ScyllaDB, three similarity functions are supported: COSINE (default), DOT_PRODUCT, and EUCLIDEAN. See Choosing a Similarity Function.

• Similarity Search — A technique for finding items in a dataset that are most similar to a query vector, using a distance or similarity measure. It is commonly used in high-dimensional vector spaces to retrieve approximate matches efficiently.

• Stemming — Reducing inflected words to a common root so that, for example, running and runs match a search for run. In ScyllaDB, stemming is performed by the language analyzers (english, german, and so on); the default standard analyzer does not stem. See Choosing an Analyzer.

• Stop Word — A very common word, such as the, of, or a, that carries little search signal and is removed during analysis. Searching for a stop word on its own matches nothing. Which words count depends on the analyzer’s language, and the simple and whitespace analyzers remove none. See How Text Is Analyzed.

• Tokenization — Splitting text into the individual terms that get indexed. How the split is made depends on the analyzer: all but whitespace split on non-alphanumeric characters, so wide-column is indexed as the two terms wide and column, while whitespace splits on whitespace only and keeps it as one term. See Choosing an Analyzer.

• USearch Index — A high-performance, in-memory vector index library developed by Unum, designed for fast approximate nearest-neighbor (ANN) search. ScyllaDB uses USearch as the underlying engine for its Vector Search Index to deliver low-latency similarity queries and efficient memory utilization.

• Vector — An ordered list of numbers (floats) representing data, such as text, images, or audio, in a way that captures its meaning or features.

• Vector Search Index — In ScyllaDB, a USearch index built on a vector column that accelerates similarity queries. Unlike traditional indexes (for exact matches or ranges), a Vector Search index is optimized for approximate nearest neighbor (ANN) lookups over high-dimensional data.

• Vector Type — A native ScyllaDB column type used to store fixed-length numeric vectors directly in a table for similarity search. See Data Types - Vectors in the ScyllaDB documentation.