What is a vector database?

As with other data types, efficiently querying a large set of vectors requires an index—and vector databases support specialized indexes for vectors. Unlike many other data types (like text or numbers), which have a single logical ordering, vectors don’t have a natural ordering that corresponds to practical use cases. Instead, the most common use case is to query for the k vectors that are closest to some other vector in terms of a distance metric like dot product, cosine similarity, or euclidean distance. This kind of query is referred to as a “k (exact) nearest neighbors” or “KNN” query.

Unfortunately, there are no general algorithms for efficient KNN queries—to be guaranteed of finding the k nearest neighbors to a given vector, q, requires computing the distance between q and every other vector. However, there are efficient algorithms for finding the k approximate nearest neighbors (“ANN”). These ANN algorithms trade off some accuracy (specifically recall, the algorithm might omit some of the actual nearest neighbors) for big improvements in speed. Since many use cases already treat the process of computing vector embeddings as somewhat imprecise, they can often tolerate some loss in recall in return for big improvements in performance.

To enable ANN queries for vectors based on distance from some other vector, a vector index is structured in a way that clusters of nearby vectors are generally grouped together. Common vector index types can be structured as a set of lists in which each list represents the vectors in a given cluster; a graph in which each vector is connected to several of their nearest neighbors; trees in which the branches correspond to subsets of the parent node’s cluster; and more. Each index type provides tradeoffs between lookup speed, recall, memory consumption, index creation time, and other factors.

Most database queries aren’t just based on semantic similarity though. For example, a user might be looking for a book whose description is similar to “a heartwarming story about a child and a dog,” but they also want to limit it to books under $20 that are available in paperback format. Special-purpose vector databases may provide some limited additional filtering capability (sometimes called “restricts”) while general purpose databases can compose rich predicates using standard languages like SQL, which can be combined with vector similarity ordering to enable very powerful, expressive queries.

The ability of vector embeddings to represent the semantic meaning of unstructured data combined with the ability for vector databases to efficiently search for nearby vectors unlocks many important use cases:

• Find messages from chat history that are relevant to the current conversation, thus enabling AI-powered chatbots to have “memory”
• Find video clips from a vast archive of sports broadcasts that match a simple description like “leaping over a defender to catch the ball and score a touchdown”
• Find products similar to something a user has purchased before and which fit within that user’s price and style preferences
• Look up documents that are relevant to a task like “replacing the filter on a vacuum” so that an AI-powered assistant can provide a relevant, factual response

Vector databases are important because they can be used to solve a variety of problems that are difficult or impossible to solve with relational predicates or text search techniques alone.

Vector databases are well-suited for generative AI applications because they make it easy to retrieve critical business and application context like relevant chat history or business-specific unstructured content to help LLMs respond to a complex user query.

They can also make it easy to search content using natural language or to query by example.