Vector indexing best practices

This page describes vector indexing best practices that optimize your vector indexes and improve approximate nearest neighbor (ANN) query results.

Tune the vector search options

The most optimal values for your vector index options depend on your use case, vector dataset, and on the query vectors. You can set and tune these values by creating a new vector index and setting the index_option_list in the CREATE VECTOR INDEX statement. You might need to perform iterative tuning to find the best values for your specific workload.

Here are some helpful guidelines to follow when picking appropriate values:

  • tree_depth (tree level): If the table you're indexing has fewer than 10 million rows, use a tree_depth of 2. Otherwise, a tree_depth of 3 supports tables of up to about 10 billion rows.

  • num_leaves: Use the square root of the number of rows in the dataset. A larger value can increase vector index build time. Avoid setting num_leaves larger than the table_row_count divided by 1000 as this results in overly small leaves and poor performance.

  • num_leaves_to_search: This option specifies how many leaf nodes of the index are searched. Increasing num_leaves_to_search improves recall but also increases latency and cost. We recommend using a number that is 1% the total number of leaves defined in the CREATE VECTOR INDEX statement as the value for num_leaves_to_search. If you're using a filter clause, increase this value to widen the search.

If acceptable recall is achieved, but the cost of querying is too high, resulting in low maximum QPS, try increasing num_leaves by following these steps:

  1. Set num_leaves to some multiple k of its original value (for example, 2 * sqrt(table_row_count)).
  2. Set num_leaves_to_search to be the same multiple k of its original value.
  3. Experiment with reducing num_leaves_to_search to improve cost and QPS while maintaining recall.

Improve recall

To improve recall, consider tuning the num_leaves_to_search value or rebuilding your vector index.

If the num_leaves_to_search value is too small, you might find it more challenging to find the nearest neighbors for some query vectors. Creating a new vector index with an increased num_leaves_to_search value can help improve recall by searching more leaves. Recent queries might contain more of these challenging vectors.

Rebuild the vector index

The tree structure of the vector index is optimized for the dataset at the time of creation, and is static thereafter. Therefore, if significantly different vectors are added after creating the initial vector index, then the tree structure might be sub-optimal, leading to poorer recall.

To rebuild your vector index without downtime:

  1. Create a new vector index on the same embedding column as the current vector index, updating parameters (for example, OPTIONS) as appropriate.
  2. After the index creation completes, use the FORCE_INDEX hint to point at the new index to update the vector search query. This ensures that the query uses the new vector index. You might also need to retune num_leaves_to_search in your new query.
  3. Drop the outdated vector index.

What's next