Ground with Google Search

This page explains how to ground a model's responses using Google Search, which uses publicly-available web data.

Grounding with Google Search

If you want to connect your model with world knowledge, a wide possible range of topics, or up-to-date information on the Internet, then use Grounding with Google Search.

Grounding with Google Search supports dynamic retrieval that gives you the option to generate grounded responses with Google Search. Therefore, the dynamic retrieval configuration evaluates whether a prompt requires knowledge about recent events and enables Grounding with Google Search. For more information, see Dynamic retrieval.

To learn more about model grounding in Vertex AI, see the Grounding overview.

Supported models

This section lists the models that support grounding with Search. To explore how each model generates grounded responses, follow these instructions:

1. Try a model listed in this table in the Google Cloud console.

2. Click the Grounding toggle to the on position.

Supported languages

For a listed of supported languages, see Languages.

Ground your model with Google Search

Use the following instructions to ground a model with publicly available web data.

Dynamic retrieval

You can use dynamic retrieval in your request to choose when to turn off grounding with Google Search. This is useful when the prompt doesn't require an answer grounded in Google Search and the supported models can provide an answer based on their knowledge without grounding. This helps you manage latency, quality, and cost more effectively.

Before you invoke the dynamic retrieval configuration in your request, understand the following terminology:

• Prediction score: When you request a grounded answer, Vertex AI assigns a prediction score to the prompt. The prediction score is a floating point value in the range [0,1]. Its value depends on whether the prompt can benefit from grounding the answer with the most up-to-date information from Google Search. Therefore, a prompt that requires an answer grounded in the most recent facts on the web, has a higher prediction score. A prompt for which a model-generated answer is sufficient has a lower prediction score.

  Here are examples of some prompts and their prediction scores.

  Prompt	Prediction score	Comment
  "Write a poem about peonies"	0.13	The model can rely on its knowledge and the answer doesn't need grounding
  "Suggest a toy for a 2yo child"	0.36	The model can rely on its knowledge and the answer doesn't need grounding
  "Can you give a recipe for an asian-inspired guacamole?"	0.55	Google Search can give a grounded answer, but grounding isn't strictly required; the model knowledge might be sufficient
  "What's Agent Builder? How is grounding billed in Agent Builder?"	0.72	Requires Google Search to generate a well-grounded answer
  "Who won the latest F1 grand prix?"	0.97	Requires Google Search to generate a well-grounded answer

• Threshold: In your request, you can specify a dynamic retrieval configuration with a threshold. The threshold is a floating point value in the range [0,1] and defaults to 0.7. If the threshold value is zero, the response is always grounded with Google Search. For all other values of threshold, the following is applicable:

  • If the prediction score is greater than or equal to the threshold, the answer is grounded with Google Search. A lower threshold implies that more prompts have responses that are generated using Grounding with Google Search.
  • If the prediction score is less than the threshold, the model might still generate the answer, but it isn't grounded with Google Search.

To find a good threshold that suits your business needs, you can create a representative set of queries that you expect to encounter. Then you can sort the queries according to the prediction score in the response and select a good threshold for your use case.

Considerations

• To use grounding with Google Search, you must enable Google Search Suggestions. See more at Google Search Suggestions.

• For ideal results, use a temperature of 0.0. To learn more about setting this config, see the Gemini API request body from the model reference.

• Grounding with Google Search has a limit of one million queries per day. If you require more queries, contact Google Cloud support for assistance.

• Only the Gemini 1.0 and Gemini 1.5 models support dynamic retrieval. The Gemini 2.0 models don't support dynamic retrieval.

  const {VertexAI} = require('@google-cloud/vertexai');

  // Initialize Vertex with your Cloud project and location
  const vertex_ai = new VertexAI({project: PROJECT_ID, location: LOCATION});
  const model = MODEL_ID;

  // Instantiate the models
  const generativeModel = vertex_ai.preview.getGenerativeModel({
    model: model,
    generationConfig: {
      'maxOutputTokens': 8192,
      'temperature': 1,
      'topP': 0.95,
    },
    safetySettings: [
      {
        'category': 'HARM_CATEGORY_HATE_SPEECH',
        'threshold': 'OFF',
      },
      {
        'category': 'HARM_CATEGORY_DANGEROUS_CONTENT',
        'threshold': 'OFF',
      },
      {
        'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT',
        'threshold': 'OFF',
      },
      {
        'category': 'HARM_CATEGORY_HARASSMENT',
        'threshold': 'OFF',
      }
    ],
    tools: [
      {
        googleSearchRetrieval: {
          dynamicRetrievalConfig: {
              mode: "MODE_DYNAMIC",
              dynamicThreshold: DYNAMIC_THRESHOLD
          }
        },
      },
    ],
  });

  async function generateContent() {
    const req = {
      contents: [
        {role: 'user', parts: [{text: `TEXT`}]}
      ],
    };

    const streamingResp = await generativeModel.generateContentStream(req);

    for await (const item of streamingResp.stream) {
      process.stdout.write('stream chunk: ' + JSON.stringify(item) + '\n');
    }

    process.stdout.write('aggregated response: ' + JSON.stringify(await streamingResp.response));
  }

  generateContent();

import vertexai

from vertexai.generative_models import (
    GenerationConfig,
    GenerativeModel,
    Tool,
    grounding,
)

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

model = GenerativeModel("gemini-1.5-flash-001")

# Use Google Search for grounding
tool = Tool.from_google_search_retrieval(
    grounding.GoogleSearchRetrieval(
        # Optional: For Dynamic Retrieval
        dynamic_retrieval_config=grounding.DynamicRetrievalConfig(
            dynamic_threshold=0.7,
        )
    )
)

prompt = "When is the next total solar eclipse in US?"
response = model.generate_content(
    prompt,
    tools=[tool],
    generation_config=GenerationConfig(
        temperature=0.0,
    ),
)

print(response.text)
# Example response:
# The next total solar eclipse visible from the contiguous United States will be on **August 23, 2044**.

POST https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:generateContent

{
  "contents": [{
    "role": "user",
    "parts": [{
      "text": "TEXT"
    }]
  }],
  "tools": [{
    "googleSearchRetrieval": {
      "dynamicRetrievalConfig": {
        "mode": "MODE_DYNAMIC",
        "dynamicThreshold": DYNAMIC_THRESHOLD
      }
    }
  }],
  "model": "projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID"
}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:generateContent"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:generateContent" | Select-Object -Expand Content

{
   "candidates": [
     {
       "content": {
         "role": "model",
         "parts": [
           {
             "text": "Chicago weather changes rapidly, so layers let you adjust easily. Consider a base layer, a warm mid-layer (sweater-fleece), and a weatherproof outer layer."
           }
         ]
       },
       "finishReason": "STOP",
       "safetyRatings":[
       "..."
    ],
       "groundingMetadata": {
         "webSearchQueries": [
           "What's the weather in Chicago this weekend?"
         ],
         "searchEntryPoint": {
            "renderedContent": "....................."
         }
         "groundingSupports": [
            {
              "segment": {
                "startIndex": 0,
                "endIndex": 65,
                "text": "Chicago weather changes rapidly, so layers let you adjust easily."
              },
              "groundingChunkIndices": [
                0
              ],
              "confidenceScores": [
                0.99
              ]
            },
          ]
          "retrievalMetadata": {
              "webDynamicRetrievalScore": 0.96879
            }
       }
     }
   ],
   "usageMetadata": { "..."
   }
 }

Understand your response

If your model prompt successfully grounds to Google Search from the Vertex AI Studio or from the API, then the responses include metadata with source links (web URLs). However, there are several reasons this metadata might not be provided, and the prompt response won't be grounded. These reasons include low source relevance or incomplete information within the model's response.

Citations

Displaying citations is highly recommended. They aid users in validating responses from publishers themselves and add avenues for further learning.

Citations for responses from Google Search sources should be shown both inline and in aggregate. See the following image as a suggestion on how to do this.

Use of alternative search engine options

Customer's use of Grounding with Google Search does not prevent Customer from offering alternative search engine options, making alternative search options the default option for Customer Applications, or displaying their own or third party search suggestions or search results in Customer Applications, provided that that any such non-Google Search services or associated results are displayed separately from the Grounded Results and Search Suggestions and can't reasonably be attributed to, or confused with results provided by, Google.

What's next

• To learn how to send chat prompt requests, see Multiturn chat.
• To learn about responsible AI best practices and Vertex AI's safety filters, see Safety best practices.
• To learn how to ground the PaLM models, see Grounding in Vertex AI.