Grounding with your data

Stay organized with collections Save and categorize content based on your preferences.

This page explains how you can ground responses by using your data from Vertex AI Search (Preview).

Grounding Gemini to your data

If you want to do retrieval-augmented generation (RAG), connect your model to your website data or your sets of documents, then use Grounding with Vertex AI Search.

Grounding to your data supports a maximum of 10 Vertex AI Search data sources and can be combined with Grounding with Google Search.

Supported models

This section lists the models that support grounding with your data. To explore how each model generates grounded responses, follow these instructions:

1. Try a model listed in this table in the Google Cloud console.

2. Click the Grounding: Your data toggle to the on position.

3. Click Customize and a Customize Grounding pane displays.

4. Select Vertex AI Search.

5. In the Grounding with Vertex AI Search section, enter the path of the Vertex AI datastore. If you don't have a Vertex AI data store, create a data store. For more information, see Create a data store.

6. Click Save.

Prerequisites

Before you can ground model output to your data, do the following:

1. Enable AI Applications and activate the API.
2. Create a AI Applications data source and application.

See the Introduction to Vertex AI Search for more.

Enable AI Applications

1. In the Google Cloud console, go to the Agent Builder page.

   Agent Builder

2. Read and agree to the terms of service, then click Continue and activate the API.

AI Applications is available in the global location, or the eu and us multi-region. To learn more, see AI Applications locations

Create a data store in AI Applications

To create a data store in AI Applications, you can choose to ground with website data or documents.

Generate grounded responses with your data store

Use the following instructions to ground a model with your data. A maximum of 10 data stores is supported.

If you don't know your data store ID, follow these steps:

1. In the Google Cloud console, go to the AI Applications page and in the navigation menu, click Data stores.

   Go to the Data stores page

2. Click the name of your data store.

3. On the Data page for your data store, get the data store ID.

from google import genai
from google.genai.types import (
    GenerateContentConfig,
    HttpOptions,
    Retrieval,
    Tool,
    VertexAISearch,
)

client = genai.Client(http_options=HttpOptions(api_version="v1"))

# Load Data Store ID from Vertex AI Search
# datastore = "projects/111111111111/locations/global/collections/default_collection/dataStores/data-store-id"

response = client.models.generate_content(
    model="gemini-2.0-flash-001",
    contents="How do I make an appointment to renew my driver's license?",
    config=GenerateContentConfig(
        tools=[
            # Use Vertex AI Search Tool
            Tool(
                retrieval=Retrieval(
                    vertex_ai_search=VertexAISearch(
                        datastore=datastore,
                    )
                )
            )
        ],
    ),
)

print(response.text)
# Example response:
# 'The process for making an appointment to renew your driver's license varies depending on your location. To provide you with the most accurate instructions...'

POST https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:generateContent

{
  "contents": [{
    "role": "user",
    "parts": [{
      "text": "TEXT"
    }]
  }],
  "tools": [{
    "retrieval": {
      "vertexAiSearch": {
        "datastore": projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID
      }
    }
  }],
  "model": "projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID"
}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:generateContent"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:generateContent" | Select-Object -Expand Content

{
  "candidates": [
    {
      "content": {
        "role": "model",
        "parts": [
          {
            "text": "You can make an appointment on the website https://dmv.gov/"
          }
        ]
      },
      "finishReason": "STOP",
      "safetyRatings": [
        "..."
      ],
      "groundingMetadata": {
        "retrievalQueries": [
          "How to make appointment to renew driving license?"
        ],
        "groundingChunks": [
          {
            "retrievedContext": {
              "uri": "https://vertexaisearch.cloud.google.com/grounding-api-redirect/AXiHM.....QTN92V5ePQ==",
              "title": "dmv"
            }
          }
        ],
        "groundingSupport": [
          {
            "segment": {
              "startIndex": 25,
              "endIndex": 147
            },
            "segment_text": "ipsum lorem ...",
            "supportChunkIndices": [1, 2],
            "confidenceScore": [0.9541752, 0.97726375]
          },
          {
            "segment": {
              "startIndex": 294,
              "endIndex": 439
            },
            "segment_text": "ipsum lorem ...",
            "supportChunkIndices": [1],
            "confidenceScore": [0.9541752, 0.9325467]
          }
        ]
      }
    }
  ],
  "usageMetadata": {
    "..."
  }
}

Understand your response

The response from both APIs include the LLM-generated text, which is called a candidate. If your model prompt successfully grounds to your Elasticsearch data source, then the responses include grounding metadata, which identifies the parts of the response that were derived from your Elasticsearch data. However, there are several reasons this metadata might not be provided, and the prompt response won't be grounded. These reasons include low-source relevance or incomplete information within the model's response.

The following is a breakdown of the output data:

• Role: Indicates the sender of the grounded answer. Because the response always contains grounded text, the role is always model.
• Text: The grounded answer generated by the LLM.
• Grounding metadata: Information about the grounding source, which contains the following elements:
  • Grounding chunks: A list of results from your Elasticsearch index that support the answer.
  • Grounding supports: Information about a specific claim within the answer that can be used to show citations:
  • Segment: The part of the model's answer that is substantiated by a grounding chunk.
  • Grounding chunk index: The index of the grounding chunks in the grounding chunks list that corresponds to this claim.
  • Confidence scores: A number from 0 to 1 that indicates how grounded the claim is in the provided set of grounding chunks.

What's next

• To learn how to send chat prompt requests, see Multiturn chat.
• To learn about responsible AI best practices and Vertex AI's safety filters, see Safety best practices.