Best practices for tuning ScaNN indexes

The recommended parameters for theScaNN index will differ depending on whether you choose to build a two-level and a three-level tree index. This page provides recommendations about how to tune index parameters for optimal balance between recall and QPS.

ScaNN index creation

For more information view the ScaNN Index reference.

Two-level tree index

To apply recommendations to help you find the optimal values of num_leaves and num_leaves_to_search for your dataset, follow these recommended steps:

  1. To create the ScaNN index optimized for the following cases set the num_leaves parameter to the following value, where rows is the number of rows in the indexed table:
    • balanced index build time and quality set num_leaves to sqrt(rows).
    • quality set num_leaves to rows/100.
  2. Run your test queries, increasing the value of scann.num_of_leaves_to_search, until you achieve your target recall range–for example, 95%. For more information about analyzing your queries, see Analyze your queries.
  3. Take note of the ratio between scann.num_leaves_to_search and num_leaves that will be used in subsequent steps. This ratio provides an approximation around the dataset that will help you achieve your target recall.

    If you are working with high dimension vectors (500 dimensions or higher) and want to improve recall, then try tuning the value of scann.pre_reordering_num_neighbors. The default value is set to the value 500 * K where K is the limit that you set in your query.
  4. If your QPS is too low after your queries achieve a target recall, then follow these steps:
    1. Recreate the index, increasing the value of num_leaves and scann.num_leaves_to_search according to the following guidance:
      • Set num_leaves to a larger factor of the square root of your row count. For example, if the index has num_leaves set to the square root of your row count, try setting it to double the square root. If the value is already double, then try setting it to triple the square root.
      • Increase scann.num_leaves_to_search as needed to maintain its ratio with num_leaves, which you noted in Step 3.
      • Set num_leaves to a value less than or equal to the row count divided by 100.
    2. Run the test queries again. While you're running the test queries, experiment with reducing scann.num_leaves_to_search, finding a value that increases QPS while keeping your recall high. Try different values of scann.num_leaves_to_search without rebuilding the index.
  5. Repeat Step 4 until both the QPS and the recall range have reached acceptable values.

Three-level tree index

In addition to the recommendations for the two-level tree ScaNN index, use the following guidance.

To apply recommendations to find the optimal value of num_leaves and max_num_levels index parameters, follow these steps:

  1. Create the ScaNN index with the following num_leaves and max_num_levels combinations based on your performance goals:

    • balance index build time & quality: Set max_num_levels as 2 and num_leaves as power(rows, ⅔).
    • optimize for quality: Set max_num_levels as 2 and num_leaves as rows/100.

  2. Run your test queries. For more information about analyzing queries, see Analyze your queries.

  3. Take note of the ratio between scann.num_leaves_to_search and num_leaves that will be used in subsequent steps. This ratio provides an approximation around the dataset that will help you achieve your target recall.

If you are working with high dimension vectors (500 dimensions or higher) and want to improve recall, then try tuning the value of scann.pre_reordering_num_neighbors. The default value is set to the value 500 * K where K is the limit that you set in your query.

  1. If your QPS is too low after your queries achieve a target recall, then follow these steps:

    • Recreate the index, increasing the value of num_leaves and scann.num_leaves_to_search according to the following guidance:
    • Set num_leaves to a larger factor of the power(rows, ⅔). For example, if the index has num_leaves set to the power(rows, ⅔), try setting it to double the power(rows, ⅔). If the value is already double, then try setting it to triple the power(rows, ⅔).
    • Increase scann.num_leaves_to_search as needed to maintain its ratio with num_leaves, which you noted in Step 3.
    • Set num_leaves to a value less than or equal to rows/100.
    • Run the test queries again. While you're running the test queries, experiment with reducing scann.num_leaves_to_search, finding a value that increases QPS while keeping your recall high. Try different values of scann.num_leaves_to_search without rebuilding the index.

  2. Repeat Step 4 until both the QPS and the recall range have reached acceptable values.

Index maintenance

If your table is prone to frequent updates or insertions, then we recommend periodically reindexing the existing ScaNN index in order to improve the recall accuracy. You can monitor index metrics to view changes in vector distributions or vector mutations since the index was built, and then reindex accordingly. For more information about metrics, see View Vector index metrics.

What's next