查詢公開索引以取得最近鄰

建立並部署索引後,您可以執行查詢來取得最近鄰。

以下列舉幾個比對查詢的範例,說明如何使用 k 近鄰演算法 (k-NN) 找出最鄰近的項目。

公開端點的查詢範例

Python 適用的 Vertex AI SDK

如要瞭解如何安裝或更新 Python 適用的 Vertex AI SDK,請參閱「安裝 Python 適用的 Vertex AI SDK」。 詳情請參閱 Vertex AI SDK for Python API 參考說明文件

def vector_search_find_neighbors(
    project: str,
    location: str,
    index_endpoint_name: str,
    deployed_index_id: str,
    queries: List[List[float]],
    num_neighbors: int,
) -> List[
    List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]
]:
    """Query the vector search index.

    Args:
        project (str): Required. Project ID
        location (str): Required. The region name
        index_endpoint_name (str): Required. Index endpoint to run the query
        against.
        deployed_index_id (str): Required. The ID of the DeployedIndex to run
        the queries against.
        queries (List[List[float]]): Required. A list of queries. Each query is
        a list of floats, representing a single embedding.
        num_neighbors (int): Required. The number of neighbors to return.

    Returns:
        List[List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]] - A list of nearest neighbors for each query.
    """
    # Initialize the Vertex AI client
    aiplatform.init(project=project, location=location)

    # Create the index endpoint instance from an existing endpoint.
    my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(
        index_endpoint_name=index_endpoint_name
    )

    # Query the index endpoint for the nearest neighbors.
    return my_index_endpoint.find_neighbors(
        deployed_index_id=deployed_index_id,
        queries=queries,
        num_neighbors=num_neighbors,
    )

指令列

您可以在「部署」中找到下列 publicEndpointDomainName,格式為 <number>.<region>-<number>.vdb.vertexai.goog


  $ curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer `gcloud auth print-access-token`" https://1957880287.us-central1-181224308459.vdb.vertexai.goog/v1/projects/181224308459/locations/us-central1/indexEndpoints/3370566089086861312:findNeighbors -d '{deployed_index_id: "test_index_public1", queries: [{datapoint: {datapoint_id: "0", feature_vector: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}, neighbor_count: 5}]}'
  

這個 curl 範例說明如何從 http(s) 用戶端呼叫,雖然公開端點支援 RESTful 和 grpc_cli 的雙重通訊協定。


  $ curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer `gcloud auth print-access-token`" https://1957880287.us-central1-181224308459.vdb.vertexai.goog/v1/projects/${PROJECT_ID}/locations/us-central1/indexEndpoints/${INDEX_ENDPOINT_ID}:readIndexDatapoints -d '{deployed_index_id:"test_index_public1", ids: ["606431", "896688"]}'
  

以下 curl 範例說明如何使用符記和數字限制進行查詢。


  $ curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer `gcloud auth print-access-token`"  https://${PUBLIC_ENDPOINT_DOMAIN}/v1/projects/${PROJECT_ID}/locations/${LOCATION}/indexEndpoints/${INDEX_ENDPOINT_ID}:findNeighbors -d '{deployed_index_id:"${DEPLOYED_INDEX_ID}", queries: [{datapoint: {datapoint_id:"x", feature_vector: [1, 1], "sparse_embedding": {"values": [111.0,111.1,111.2], "dimensions": [10,20,30]}, numeric_restricts: [{namespace: "int-ns", value_int: -2, op: "GREATER"}, {namespace: "int-ns", value_int: 4, op: "LESS_EQUAL"}, {namespace: "int-ns", value_int: 0, op: "NOT_EQUAL"}], restricts: [{namespace: "color", allow_list: ["red"]}]}}]}'
  

控制台

請按照下列操作說明,透過控制台查詢部署至公開端點的索引。

  1. 在 Google Cloud 控制台的 Vertex AI 專區中,前往「部署及使用」專區。選取「向量搜尋」

    前往 Vector Search

  2. 選取要查詢的索引。「索引資訊」頁面隨即開啟。
  3. 向下捲動至「已部署的索引」部分,然後選取要查詢的已部署索引。「已部署的索引資訊」頁面隨即開啟。
  4. 在「查詢索引」部分,選取要依據密集嵌入值、稀疏嵌入值、混合嵌入值 (密集和稀疏嵌入) 或特定資料點查詢。
  5. 輸入所選查詢類型的查詢參數。舉例來說,如果您要查詢稠密嵌入,請輸入要查詢的嵌入向量。
  6. 使用提供的 curl 指令執行查詢,或透過 Cloud Shell 執行。
  7. 如果使用 Cloud Shell,請選取「Run in Cloud Shell」
  8. 在 Cloud Shell 中執行。
  9. 結果會傳回最相近的項目。

混合查詢

混合搜尋會根據關鍵字搜尋和語意搜尋的組合,使用密集和稀疏嵌入來進行搜尋。

Python 適用的 Vertex AI SDK

如要瞭解如何安裝或更新 Python 適用的 Vertex AI SDK,請參閱「安裝 Python 適用的 Vertex AI SDK」。 詳情請參閱 Vertex AI SDK for Python API 參考說明文件

def vector_search_find_neighbors_hybrid_queries(
    project: str,
    location: str,
    index_endpoint_name: str,
    deployed_index_id: str,
    num_neighbors: int,
) -> List[
    List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]
]:
    """Query the vector search index using example hybrid queries.

    Args:
        project (str): Required. Project ID
        location (str): Required. The region name
        index_endpoint_name (str): Required. Index endpoint to run the query
        against.
        deployed_index_id (str): Required. The ID of the DeployedIndex to run
        the queries against.
        num_neighbors (int): Required. The number of neighbors to return.

    Returns:
        List[List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]] - A list of nearest neighbors for each query.
    """
    # Initialize the Vertex AI client
    aiplatform.init(project=project, location=location)

    # Create the index endpoint instance from an existing endpoint.
    my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(
        index_endpoint_name=index_endpoint_name
    )

    # Query hybrid datapoints, sparse-only datapoints, and dense-only datapoints.
    hybrid_queries = [
        aiplatform.matching_engine.matching_engine_index_endpoint.HybridQuery(
            dense_embedding=[1, 2, 3],
            sparse_embedding_dimensions=[10, 20, 30],
            sparse_embedding_values=[1.0, 1.0, 1.0],
            rrf_ranking_alpha=0.5,
        ),
        aiplatform.matching_engine.matching_engine_index_endpoint.HybridQuery(
            dense_embedding=[1, 2, 3],
            sparse_embedding_dimensions=[10, 20, 30],
            sparse_embedding_values=[0.1, 0.2, 0.3],
        ),
        aiplatform.matching_engine.matching_engine_index_endpoint.HybridQuery(
            sparse_embedding_dimensions=[10, 20, 30],
            sparse_embedding_values=[0.1, 0.2, 0.3],
        ),
        aiplatform.matching_engine.matching_engine_index_endpoint.HybridQuery(
            dense_embedding=[1, 2, 3]
        ),
    ]

    return my_index_endpoint.find_neighbors(
        deployed_index_id=deployed_index_id,
        queries=hybrid_queries,
        num_neighbors=num_neighbors,
    )

含有篩選和擁擠情況的查詢

篩選向量比對結果:可將近鄰結果限制在特定類別。篩選器也可以指定要從結果中排除的類別。

每個擁擠鄰居限制可限制索引資料中任何單一擁擠標記傳回的結果數量,藉此增加結果多樣性。

Python 適用的 Vertex AI SDK

如要瞭解如何安裝或更新 Python 適用的 Vertex AI SDK,請參閱「安裝 Python 適用的 Vertex AI SDK」。 詳情請參閱 Vertex AI SDK for Python API 參考說明文件

def vector_search_find_neighbors_filtering_crowding(
    project: str,
    location: str,
    index_endpoint_name: str,
    deployed_index_id: str,
    queries: List[List[float]],
    num_neighbors: int,
    filter: List[aiplatform.matching_engine.matching_engine_index_endpoint.Namespace],
    numeric_filter: List[
        aiplatform.matching_engine.matching_engine_index_endpoint.NumericNamespace
    ],
    per_crowding_attribute_neighbor_count: int,
) -> List[
    List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]
]:
    """Query the vector search index with filtering and crowding.

    Args:
        project (str): Required. Project ID
        location (str): Required. The region name
        index_endpoint_name (str): Required. Index endpoint to run the query
        against.
        deployed_index_id (str): Required. The ID of the DeployedIndex to run
        the queries against.
        queries (List[List[float]]): Required. A list of queries. Each query is
        a list of floats, representing a single embedding.
        num_neighbors (int): Required. The number of neighbors to return.
        filter (List[Namespace]): Required. A list of Namespaces for filtering
        the matching results. For example,
        [Namespace("color", ["red"], []), Namespace("shape", [], ["square"])]
        will match datapoints that satisfy "red color" but not include
        datapoints with "square shape".
        numeric_filter (List[NumericNamespace]): Required. A list of
        NumericNamespaces for filtering the matching results. For example,
        [NumericNamespace(name="cost", value_int=5, op="GREATER")] will limit
        the matching results to datapoints with cost greater than 5.
        per_crowding_attribute_neighbor_count (int): Required. The maximum
        number of returned matches with the same crowding tag.

    Returns:
        List[List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]] - A list of nearest neighbors for each query.
    """
    # Initialize the Vertex AI client
    aiplatform.init(project=project, location=location)

    # Create the index endpoint instance from an existing endpoint.
    my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(
        index_endpoint_name=index_endpoint_name
    )

    # Query the index endpoint for the nearest neighbors.
    return my_index_endpoint.find_neighbors(
        deployed_index_id=deployed_index_id,
        queries=queries,
        num_neighbors=num_neighbors,
        filter=filter,
        numeric_filter=numeric_filter,
        per_crowding_attribute_neighbor_count=per_crowding_attribute_neighbor_count,
    )

影響效能的查詢時間設定

以下查詢時間參數會影響使用向量搜尋時的延遲時間、可用性和費用。這項指引適用於大多數情況。不過,請務必嘗試設定,確保設定適用於您的用途。

如需參數定義,請參閱「索引設定參數」。

參數 關於 效能影響
approximateNeighborsCount

告知演算法從每個分割區擷取的近似結果數量。

approximateNeighborsCount 的值一律應大於 setNeighborsCount 的值。如果 setNeighborsCount 的值很小,建議將 approximateNeighborsCount 設為該值的 10 倍。如果 setNeighborsCount 值較大,則可使用較小的乘數。

這個欄位的對應 REST API 名稱為 approximate_neighbor_count

提高 approximateNeighborsCount 的值可能會對效能造成以下影響:

  • 喚回度:增加
  • 延遲:可能會增加
  • 供應情形:沒有影響
  • 成本:搜尋期間處理的資料量可能會增加,因此成本可能會增加

降低 approximateNeighborsCount 的值可能會對效能造成以下影響:

  • 喚回度:降低
  • 延遲:可能會減少
  • 供應情形:沒有影響
  • 成本:搜尋期間處理的資料較少,因此可降低成本
setNeighborCount

指定要查詢傳回的結果數量。

這個欄位的對應 REST API 名稱為 neighbor_count

在大多數用途中,值小於或等於 300 仍可維持效能。如果是較大的值,請針對特定用途進行測試。

fractionLeafNodesToSearch 控制在搜尋最近鄰點時,要造訪的分葉節點百分比。這與 leafNodeEmbeddingCount 有關,因為每個葉節節點的嵌入資料越多,每個葉節檢查的資料就越多。

這個欄位的對應 REST API 名稱為 fraction_leaf_nodes_to_search_override

增加 fractionLeafNodesToSearch 的值可能會對效能造成以下影響:

  • 喚回度:增加
  • 延遲時間:增加
  • 供應情形:沒有影響
  • 成本:延遲時間越長,機器資源占用量就會增加,因此成本可能會增加

降低 fractionLeafNodesToSearch 的值可能會對效能造成以下影響:

  • 喚回度:降低
  • 延遲時間:減少
  • 供應情形:沒有影響
  • 成本:由於延遲時間越短,所需機器資源越少,因此成本可能會降低

後續步驟

如需建立索引、將索引部署至公開端點,以及如何查詢的端對端範例,請參閱官方 Notebook:「使用 Vector Search 和 Vertex AI Embeddings for Text 處理 Stack Overflow 問題」。