Vertex AI release notes

The Gen AI evaluation service can now help you evaluate your translation models using MetricX, COMET, and BLEU metrics. To learn more about evaluating your translation models, see Evaluate translation models.

Batch predictions for Llama models on Vertex AI (MaaS) is available in Preview.

Batch prediction support for Gemini

Batch prediction is available for Gemini in General Availability (GA). Available Gemini models include Gemini 1.0 Pro, Gemini 1.5 Pro, and Gemini 1.5 Flash. To get started with batch prediction, see Get batch predictions for Gemini.

You can now fine-tune the following models from the Cloud console:

• Gemma 2
• Llama 3.1
• Mistral
• Mixtral

The Whisper large v3 and Whisper large v3 turbo models have been added to Model Garden.

The Vertex AI Gemini API SDK supports tokenization capabilities for local token counting and computation. This is a streamlined way to compute tokens locally, ensuring compatibility across different Gemini models and their tokenizers. Supported models include gemini-1.5-flash and gemini-1.5-pro . To learn more, see Count tokens.

The AI assistant in Vertex AI Studio can help you refine and generate prompts. This feature is in Preview. To learn more, see Use AI-powered prompt writing tools.

Prompt Guard and Flux were added to Model Garden.

You can deploy Hugging Face models on Google Cloud that have text embedding inference enabled or pytorch inference enabled. For more information, see the Hugging Face model deployment in the console.

Grounding: Dynamic retrieval for grounded results (GA)

Dynamic retrieval lets you choose when to turn off grounding with Google Search. This is useful when a prompt doesn't require an answer grounded in Google Search, and the supported models can provide an answer based on their knowledge without grounding. Dynamic retrieval helps you manage latency, quality, and cost more effectively.

This feature is Generally Available. For more information, see Dynamic retrieval.

Prompt templates let you to test how different prompt formats perform with different sets of prompt data. This feature is in Preview. To learn more, see Use prompt templates.

Use Gemini to directly analyze YouTube videos and publicly available media (such as images, audio, and video) by using a link. This feature is in Public Preview.

The new API parameters audioTimestamp, responseLogprob, and logprobs are in Public Preview. For more information, see API reference.

Gemini 1.5 Pro and Gemini 1.5 Flash now support multimodal input with function calling. This feature is in Preview.

The Vertex AI prompt optimizer adapts your prompts using the optimal instructions and examples to elicit the best performance from your chosen model. This feature is available in Preview. To learn more, see Optimize prompts.

Add label metadata to generateContent and streamGenerateContent API calls. For details, see Add labels to API calls.

Gen AI Evaluation Service is Generally Available. To learn more, see the Gen AI Evaluation Service overview.

Vertex AI SDK for Python supports token listing and counting for prompts without the need to make API calls. This feature is available in (Preview). For details, see List and count tokens.

New Imagen on Vertex AI image generation model and features

The Imagen 3 image generation models (imagen-3.0-generate-001 and the low-latency version imagen-3.0-fast-generate-001) are Generally Available to approved users. These models offer the following additional features:

• Additional aspect ratios (1:1, 3:4, 4:3, 9:16, 16:9)
• Digital watermark (SynthID) enabled by default
• Watermark verification
• User-configurable safety features (safety setting, person/face setting)

For more information, see Model versions and Generate images using text prompts.

The following models have been added to Model Garden:

• Gemma 2 2B: A foundation LLM by Google Deepmind.
• Qwen2: An LLM series by Alibaba Cloud.
• Phi-3: An LLM series by Microsoft.

The following models have been added to Model Garden:

• 36 Hugging Face embedding models with verified deployment settings such as BAAI/bge-m3 and intfloat/multilingual-e5-large-instruct.
• 35 Hugging Face PyTorch models with verified deployment settings such as stabilityai/stable-diffusion-2-1.

For more information, see the Hugging Face model deployment in the console.

Launched Hex-LLM for high-efficiency large language model serving. This performant TPU serving solution is based on XLA and optimized kernels to achieve high throughput and low latency.

Hex-LLM uses several parallelism strategies for multiple TPU chips, quantizations, dynamic LoRA, and more. Hex-LLM supports the following dense and sparse LLMs:

• Gemma 2B and 7B
• Gemma 2 9B and 27B
• Llama 2 7B, 13B and 70B
• Llama 3 8B and 70B
• Mistral 7B and Mixtral 8x7B

Context caching is available for Gemini 1.5 Pro. Use context caching to reduce the cost of requests that contain repeat content with high input token counts. For more information, see Context caching overview.

Controlled generation is available on Gemini 1.5 Pro and supports the JSON schema. For more information, see Control generated output.

Experiment in the Vertex AI Studio login-free

The Vertex AI Studio multi-model prompt designer can be accessed login-free. With this feature, prospective customers can use the Vertex AI Studio to test queries before deciding to sign up and create an account. To learn more about this experience, see Vertex AI Studio console experiences or to access the console directly go to Vertex AI Studio.

Gemini models support the frequencyPenalty and presencePenalty parameters. Use frequencyPenalty to control the probability of repeated text in a response. Use presencePenalty to control the probability of generating more diverse content. For more information, see Gemini model parameters.

The following models have been added to Model Garden:

• E5: A text embedding model series that can be served with a GPU or CPU.
• Instant ID: An identity preserving text-to-image generation model.
• Stable Diffusion XL lightning: A text-to-image generation model that is based on SDXL but requires fewer inference iterations.

To see a list of all available models, see Explore models in Model Garden.

Grounding Gemini with Google Search is GA

The Gemini API Grounding with Google Search feature is available in GA. This is available for Gemini 1.0 Pro models. To learn more about model grounding, see Grounding with Google Search.

Batch prediction support for Gemini

Batch prediction is available for Gemini in preview. Available Gemini models include Gemini 1.0 Pro, Gemini 1.5 Pro, and Gemini 1.5 Flash. To get started with batch prediction, see Get batch predictions for Gemini.

New stable text embedding models

The following text embedding models are available GA:

• text-embedding-004
• text-multilingual-embedding-002

For details on how to use these models, see Get text embeddings.

Meta's open weight Llama 3 model is available in the Vertex AI Model Garden.

New Imagen on Vertex AI image generation model and features

The 006 version of the Imagen 2 image generation model (imagegeneration@006) is now available. This model offers the following additional features:

• Additional aspect ratios (1:1, 3:4, 4:3, 9:16, 16:9)
• Digital watermark (SynthID) enabled by default
• Watermark verification*
• New user-configurable safety features (safety setting, person/face setting)

For more information, see Model versions and Generate images using text prompts.

* The seed field can't be used while digital watermark is enabled.

New Imagen on Vertex AI image editing model and features

The 006 version of the Imagen 2 image editing model (imagegeneration@006) is now available. This model offers the following additional features:

• Inpainting - Add or remove content from a masked area of an image
• Outpainting - Expand a masked area of an image
• Product image editing - Identify and maintain a primary product while changing the background or product position

For more information, see Model versions.

Grounding Gemini and Grounding with Google Search

The Gemini API now supports Grounding with Google Search in Preview. Currently available for Gemini 1.0 Pro models.

Generative AI on Vertex AI security control update

Security controls are available for the online prediction feature for Gemini 1.0 Pro and Gemini 1.0 Pro Vision.

New text embedding models

The following text embedding models are now in Preview.

• text-embedding-preview-0409
• text-multilingual-embedding-preview-0409

When evaluated using the MTEB benchmarks, these models produce better embeddings compared to previous versions. The new models also offer dynamic embedding sizes, which you can use to output smaller embedding dimensions, with minor performance loss, to save on computing and storage costs.

For details on how to use these models, refer to the public documentation and try out our Colab.

System instructions

System instructions are supported in Preview by the Gemini 1.0 Pro (stable version gemini-1.0-pro-002 only) and Gemini 1.5 Pro (Preview) multimodal models. Use system instructions to guide model behavior based on your specific needs and use cases. For more information, see System instructions examples.

Supervised Tuning for Gemini

Supervised tuning is available for the gemini-1.0-pro-002 model.

Online Evaluation Service

Generative AI evaluation supports online evaluation in addition to pipeline evaluation. The list of supported evaluation metrics has also expanded. See API reference and SDK reference.

Text translation

Translate text in Vertex AI Studio is available in Preview.

Model Garden supports all Text Generation Inference supported models in HuggingFace:

• Verified deployment settings for about 400 Hugging Face text generation models (including google/gemma-7b-it, meta-llama/Llama-2-7b-chat-hf, and mistralai/Mistral-7B-v0.1).
• Other Hugging Face text generation models have unverified deployment settings that are auto generated.