Migrate your application to Gemini 2 with the Gemini API in Vertex AI

For a seamless Gemini 2 migration, we recommend that you address the following concerns before you begin the migration process.

• Model retirement awareness: Note the model version support and availability timelines for older Gemini models, and make sure your migration is completed before the model you're using is retired.
• InfoSec, governance, and regulatory approvals: Proactively request the approvals you need for Gemini 2 from your information security (InfoSec), risk, and compliance stakeholders. Make sure that you cover domain-specific risk and compliance constraints, especially in heavily regulated industries such as healthcare and financial services. Note that Gemini security controls differ among Gemini 2 models.
• Location availability: See the Generative AI on Google Cloud models and partner model availability documentation, and make sure your chosen Gemini 2 model is available in the regions where you need it, or consider switching to the global endpoint.
• Modality and tokenization-based pricing differences: Check Gemini 2 pricing for all the modalities (text, code, images, speech) in your application. For more information, see generative AI pricing page. Note that Gemini 2 text input and output is priced per token, while Gemini 1 text input and output is priced per character.
• Provisioned Throughput: If needed, purchase additional Provisioned Throughput for Gemini 2 or change existing Provisioned Throughput orders.
• Supervised fine-tuning: If your Gemini application uses supervised fine-tuning, submit a new tuning job with Gemini 2. We recommend that you start with the default tuning hyperparameters instead of reusing the hyperparameter values that you used with previous Gemini versions. The tuning service has been optimized for Gemini 2. Therefore, reusing previous hyperparameter values might not yield the best results.
• Regression testing: There are three main types of regression tests involved when upgrading to Gemini 2 models:
  • Code regression tests: Regression testing from a software engineering and DevOps perspective. This type of regression test is always required.
  • Model performance regression tests: Regression testing from a data science or machine learning perspective. This means ensuring that the new Gemini 2 model provides outputs that are at least as high-quality as outputs from the current production model. Model performance regression tests are just model evaluations done as part of a change to a system or to the underlying model. Model performance regression testing further breaks down into:
    • Offline model performance testing: Assessing the quality of model outputs in a dedicated experimentation environment based on various model output quality metrics.
    • Online model performance testing: Assessing the quality of model outputs in a live online deployment based on implicit or explicit user feedback.
  • Load testing: Assessing how the application handles high volumes of inference requests. This type of regression test is required for applications that require Provisioned Throughput.

1. Prepare to repeat any relevant evaluations from when you originally built your application, along with any relevant evaluations you have done since then.
2. If you feel your existing evaluations don't appropriately cover or measure the breadth of tasks that your application performs, you should design and prepare additional evaluations.
3. If your application involves RAG, tool use, complex agentic workflows, or prompt chains, make sure that your existing evaluation data allows for assessing each component independently. If not, gather input-output examples for each component.
4. If your application is especially high-impact, or if it's part of a larger user-facing real-time system, you should include online evaluation.

If your Gemini 1.x application uses the Vertex AI SDK, consider upgrading to the Gen AI SDK. New Gemini 2 capabilities are only available in the Gen AI SDK. However, there is no need to switch to the Gen AI SDK if your application only requires capabilities that are available in the Vertex AI SDK.

If you're new to the Gen AI SDK, see the Getting started with Google Generative AI using the Gen AI SDK notebook.

pip install --upgrade google-genai

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=global
export GOOGLE_GENAI_USE_VERTEXAI=True

from google import genai
from google.genai.types import HttpOptions

client = genai.Client(http_options=HttpOptions(api_version="v1"))
response = client.models.generate_content(
model="gemini-2.5-flash",
contents="How does AI work?",
)
print(response.text)

# Example response:
# Okay, let's break down how AI works. It's a broad field, so I'll focus on the ...
# Here's a simplified overview:
# ...

 pip install --upgrade --quiet google-cloud-aiplatform 

 
import vertexai
from vertexai.generative_models import GenerativeModel

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

model = GenerativeModel("gemini-2.0-flash-001")

response = model.generate_content(
    "What's a good name for a flower shop that specializes in selling bouquets of dried flowers?"
)

print(response.text)
# Example response:
# **Emphasizing the Dried Aspect:**
# * Everlasting Blooms
# * Dried & Delightful
# * The Petal Preserve
# ...

Change your Gemini calls

Change your prediction code to use Gemini 2. At a minimum, this means changing the specific model endpoint name to a Gemini 2 model where you load your model.

The exact code change will differ depending on how you originally implemented your application, and especially whether you used the Gen AI SDK or the Vertex AI SDK.

After you make your code changes, perform code regression testing and other software tests on your code to make sure that it runs. This test is only meant to assess whether the code functions correctly. It's not meant to assess the quality of model responses.

Address breaking code changes

• Dynamic retrieval: Switch to using Grounding with Google Search. This feature requires using the Gen AI SDK; it's not supported by the Vertex AI SDK.
• Content filters: Note the default content filter settings, and change your code if it relies on a default that has changed.
• Top-K token sampling parameter: Models after gemini-1.0-pro-vision don't support changing the Top-K parameter.

Focus only on code changes in this step. You may need to make other changes, but wait until you start your evaluation, and then consider the following adjustment based on evaluation results:

• If you're switching from dynamic retrieval, you might need to experiment with system instructions to control when Google Search is used (for example, "Only generate queries for the Google Search tool if the user asks about sports. Don't generate queries for any other topic."), but wait until you evaluate before making prompt changes.
• If you used the Top-K parameter, adjust other token sampling parameters, such as Top-P, to achieve similar results.

Repeat the evaluation that you did when you originally developed and launched your application, any further offline evaluation you did after launching, and any additional evaluation you identified in step 1. If you then feel that your evaluation doesn't fully capture the breadth and depth of your application, do further evaluation.

If you don't have an automated way to run your offline evaluations, consider using the Gen AI evaluation service.

If your application uses fine-tuning, perform offline evaluation before retuning your model with Gemini 2. Gemini 2's improved output quality may mean that your application no longer requires a fine-tuned model.

If your offline evaluation shows a drop in performance with Gemini 2, iterate on your application as follows until Gemini performance matches the older model:

• Iteratively engineer your prompts to improve performance ("Hill Climbing"). If you are new to hill climbing, see the Vertex Gemini hill climbing online training. The Vertex AI prompt optimizer (example notebook) can help as well.
• If your application already relies on fine-tuning, try fine-tuning Gemini 2.
• If your application is impacted by Dynamic Retrieval and Top-K breaking changes, experiment with changing your prompt and token sampling parameters.

If your application requires a certain minimum throughput, perform load testing to make sure the Gemini 2 version of your application meets your throughput requirements.

Load testing should happen before online evaluation, because online evaluation requires exposing Gemini 2 to production traffic. Use your existing load testing instrumentation to perform this step.

If your application already meets throughput requirements, consider using Provisioned Throughput. You'll need additional short-term Provisioned Throughput to cover load testing while your existing Provisioned Throughput order continues to serve production traffic.

Only proceed to online evaluation if your offline evaluation shows adequate Gemini output quality and your application requires online evaluation.

Online evaluation is a special case of online testing. Try to use your organization's existing tools and procedures for online evaluation. For example:

• If your organization regularly conducts A/B tests, perform an A/B test that evaluates the current implementation of your application compared to the Gemini 2 version.
• If your organization regularly conducts canary deployments, be sure to do so with Gemini 2 and measure differences in user behavior.

Online evaluation can also be done by building new feedback and measurement capabilities into your application. Different feedback and measurement capabilities are appropriate for different applications. For example:

• Adding thumbs-up and thumbs-down buttons next to model outputs and comparing thumbs-up versus thumbs-down rates between an older model and Gemini 2.
• Presenting users with the older model and Gemini 2 outputs side-by-side and asking for users to pick their favorite.
• Tracking how often users override or manually adjust older model versus Gemini 2 outputs.

These kinds of feedback mechanisms often require running a Gemini 2 version of your application in parallel to your existing version. This parallel deployment is sometimes called "shadow mode" or "blue-green deployment".

If online evaluation results differ significantly from offline evaluation results, your offline evaluation is not capturing key aspects of the live environment or user experience. Use the online evaluation findings to devise a new offline evaluation to cover the gap the online evaluation exposed, and then return to step 3.

If you use Provisioned Throughput, you may need to purchase additional short-term Provisioned Throughput to continue to meet your throughput requirements for users subject to online evaluation.

Once your evaluation shows that Gemini 2 meets or exceeds performance of an older model, turn down the existing version of your application in favor of the Gemini 2 version. Follow your organization's existing procedures for production rollout.

If you're using Provisioned Throughput, change your Provisioned Throughput order to your chosen Gemini 2 model. If you're rolling out your application incrementally, use short-term Provisioned Throughput to meet throughput requirements for two different Gemini models.