Generative AI on Vertex AI Cookbook

Intro to Gemini 2.0 Flash

Get started with Gemini 2.0 in Vertex AI with the Gen AI Python SDK.

Getting Started with the Multimodal Live API

Get started with Gemini 2.0 Flash Multimodal Live API in Vertex AI using the Gen AI Python SDK

Getting Started with Gemini 2.0 Flash Thinking Mode

Get started with Gemini 2.0 Flash Thinking Mode in Vertex AI using the Gen AI Python SDK to get more detailed reasoning and thinking steps.

Intro to Prompt Engineering

Learn the essentials and best practices of prompt engineering.

Function Calling with Gemini

Connect Gemini to external tools using function calling.

Grounding with Gemini

Connect Gemini to real-world data from Google Search or Vertex AI Search to improve response quality.

Batch Prediction with Gemini

Use Batch Prediction to run inference on a large number of examples.

Long Context Window

Use the Long Context Window to process large amounts of multimodal data.

Intro to Context Caching

Use context caching to store frequently used data.

Intro to Controlled Generation with the Gemini API

Learn to control Gemini API output formats for easier data processing.

Call Gemini by using the OpenAI Library

Learn how to call Gemini using Chat Completions.

Creating Marketing Assets using Gemini 2.0 Flash

Learn how to combine the multimodal capabilities of Gemini and Grounding with Google Search to create a marketing campaign brief and marketing assets.

Get started with Chirp 2 for Speech-to-Text

Learn about how to use Chirp 2, the latest generation of Google's multilingual Automatic Speech Recognition models.

Intro to Imagen 3 Image Generation

Use Imagen 3 to create photorealistic images.

Imagen 3 Image Editing

Use Imagen 3 to edit photorealistic images using inpainting, outpainting, and Product Image Editing.

Get Started with Text Embeddings + Vertex AI Vector Search

Use AI's multitool, Embeddings, and Vertex AI Vector Search perform semantic matching.

Intro to LangGraph with Gemini

Learn how to combine LangGraph's workflow capabilities with Gemini's language understanding and generation skills to streamline and automate complex financial analysis tasks.

Create custom podcast episodes

Use Gemini, LangGraph, and Text-to-Speech to create custom podcast episodes.

Storytelling with Gemini and Text-to-Speech

Use Gemini and Text-to-Speech to create and read a story with multiple characters.

Analyze a codebase

Use Gemini to generate code, summarize a codebase, debug, improve code, and assess code.

LLM Security for developers

Learn about prompt injection attacks and how to mitigate them.

Gen AI Evaluation Service

Evaluate Gemini responses using metrics and custom datasets.

Supervised Fine-tuning with Gemini

Learn how to fine-tune Gemini for specific tasks.

Intro to Vertex AI RAG Engine

Build custom RAG workflows with Gemini and Vertex AI RAG Engine.

Document Processing with Gemini

Use Gemini to process documents for classification, extraction, and summarization.

Patents Document Understanding with Gemini 2.0

Use Gemini 2.0 Flash to process patent documents using classification, entity extraction, and object detection.

Intro to the Google Gen AI SDK

Intro to the Google Gen AI SDK.

Gemini with Multimodal Data

How to prompt Gemini with multimodal data (text, documents, images, video, and audio).

Getting started with Gemini 2.0 in Vertex AI using REST API

Use Gemini 2.0 with REST/curl.

Prompt Engineering, Evaluation, and Prompt Templating

Use Gen AI Evaluation Service SDK for prompt engineering and evaluation.

Intro to Multimodal Embeddings

Learn about multimodal embeddings.

Intro to Embeddings Tuning

Learn how to tune embeddings models.

Task Type Embeddings

Learn how to get better embeddings for your specific task.

Hybrid Search with Vertex AI Vector Search

Learn how to use hybrid search with Vertex AI Vector Search combining semantic & keyword search.

AI Agents for Engineers (Evolution of AI Agents)

This notebook demonstrates 3 different approaches to generating essays using the Gemini API, Zero-Shot, Step-by-Step with LangChain, and Iterative with LangGraph.

Hugging Face DLCs: Serving Gemma with Text Generation Inference (TGI) on Vertex AI

Learn how to deploy Google Gemma from the Hugging Face Hub on Vertex AI using the Hugging Face Deep Learning Container (DLC) for Text Generation Inference (TGI).

Cloud Run GPU Inference: Gemma 2 RAG Q&A with Ollama and LangChain

Learn how to deploy Gemma 2 in Cloud Run to build a simple RAG Q&A application.

Hugging Face DLCs: Serving PaliGemma using Pytorch Inference on Vertex AI with Custom Handler

Learn how to deploy Google PaliGemma from the Hugging Face Hub on Vertex AI using the Hugging Face Deep Learning Container (DLC) for Pytorch Inference in combination with a custom handler.

Hugging Face DLCs: Fine-tuning Gemma with Transformer Reinforcement Learning (TRL) on Vertex AI

Learn how to fine-tune Gemma with Transformer Reinforcement Learning (TRL) on Vertex AI.

Guess who or what app using Hugging Face Deep Learning container model on Vertex AI

Create a"Guess who or what" application using Vertex AI, Hugging Face Deep Learning container, an image generation open model, and Gemini to solve and visualize riddles.

Visualizing embedding similarity from text documents using t-SNE plots

Visualize embedding similarity from text documents using t-SNE plots.

Vertex AI Vector Search Quickstart

Learn how to use Vertex AI Vector Search to find similar text documents.

Anomaly Detection of Infrastructure Logs using Gemini and BigQuery Vector Search

Learn how to large volumes of infrastructure logs using Gemini, vector embeddings and BigQuery Vector Search to perform anomaly detection.

Log Anomaly Detection & Investigation with Text Embeddings + BigQuery Vector Search

Learn how to large volumes of audit logs using Gemini, vector embeddings and BigQuery Vector Search to perform anomaly detection.

Working with Data Structures and Schemas in Gemini Function Calling

Learn how to use Gemini Function Calling with data structures and schemas.

Working with Parallel Function Calls and Multiple Function Responses in Gemini

Learn how to use parallel function calls and multiple function responses in Gemini.

Introduction to ReAct Agents with Gemini & Function Calling

Learn how to use ReAct Agents with Gemini and Function Calling.

Forced Function Calling with Tool Configurations in Gemini

Learn how to use forced function calling with tool configurations in Gemini.

Using Gemini Function Calling to Get Real-Time Company News and Insights

Learn how to use Gemini Function Calling to get real-time company news and insights.

Multimodal Function Calling with the Gemini API & Python SDK

Learn how to use Gemini Multimodal Function Calling.

Advanced RAG Techniques - Vertex RAG Engine Retrieval Quality Evaluation and Hyperparameters Tuning

Learn about advanced RAG techniques with evaluation and hyperparameter tuning.

Vertex AI RAG Engine with Pinecone

Learn how to use Pinecone with Vertex AI RAG Engine.

Vertex AI RAG Engine with Weaviate

Learn how to use Weaviate with Vertex AI RAG Engine.

Vertex AI RAG Engine with Vertex AI Feature Store

Learn how to use Vertex AI Feature Store with Vertex AI RAG Engine.

Vertex AI RAG Engine with Vertex AI Vector Search

Learn how to use Vertex AI Vector Search with Vertex AI RAG Engine.

Vertex AI RAG Engine with Vertex AI Search

Learn how to use Vertex AI Search with Vertex AI RAG Engine.

Debugging and Optimizing Agents: A Guide to Tracing in Reasoning Engine

Learn how to use Tracing in Reasoning Engine.

Deploying a RAG Application with AlloyDB to LangChain on Vertex AI

Building a Conversational Search Agent with Reasoning Engine and RAG on Vertex AI Search

Intro to Building and Deploying an Agent with Reasoning Engine in Vertex AI

Building a Multi-Agent RAG Application with LangGraph and Reasoning Engine

Deploying a RAG Application with Cloud SQL for PostgreSQL to LangChain on Vertex AI

Building and Deploying a Google Maps API Agent with Reasoning Engine

Building and Deploying a LangGraph Application with Reasoning Engine in Vertex AI

Gemini 1.5: A workshop in multimodal use cases

Video Analysis with Gemini

YouTube Video Analysis with Gemini

Building a Multimodal Chatbot for Warranty Claims using Gemini and Vector Search in Vertex AI

Multimodal Retrieval Augmented Generation (RAG) using Gemini API in Vertex AI

LlamaIndex with Vertex AI Vector Search to perform question answering RAG

Multimodal Retrieval Augmented Generation (RAG) with Gemini, Vertex AI Vector Search, and LangChain

Small-to-big Retrieval-Augmented Generation

ReAct (Reasoning + Acting) + Custom tool for Healthcare NL API + Gemini 1.5 Pro + LangChain

Using Gemini in Education

AI Quick Build Experience

Code Retrieval Augmented Generation (RAG) with Gemini API

Product listing generation with Gemini 1.5 Pro

Multimodal retail recommendation: using Gemini to recommend items based on images and image reasoning

Sheet Music Analysis with Gemini

Analyzing movie posters in BigQuery with Gemini 1.5 Flash

Getting Started with LangChain 🦜️🔗 + Gemini API in Vertex AI

Building a Weather Agent with AutoGen and Gemini

Vertex AI Prompt Optimizer Notebook UI

Vertex AI Prompt Optimizer - Tool usage

Vertex AI Prompt Optimizer - Custom metric

Intro to Vertex AI Prompt Optimizer

Text Summarization with Generative Models on Vertex AI

Ideation with Generative Models on Vertex AI

Chain of Thought & ReAct

Question Answering with Generative Models on Vertex AI

Text Classification with Generative Models on Vertex AI

Text Extraction with Generative Models on Vertex AI

Supervised Fine Tuning with Gemini 1.5 Flash for Change Detection

Vertex AI Supervised Tuning Token Count and Cost Estimation

Supervised Fine-Tuning with Gemini 1.5 Flash for Q&A

Supervised Fine Tuning with Gemini 1.5 Pro for Image Captioning

Use Gen AI Evaluation SDK to Evaluate Models in Vertex AI Studio, Model Garden, and Model Registry

Evaluate Generated Answers from RAG using Rapid Evaluation and Dataflow ML with Vertex AI Pipelines

Enhancing Quality and Explainability with Vertex AI Evaluation

Evaluate and Compare Gen AI Model Settings

Bring-Your-Own-Autorater using Custom Metrics

Evaluate a Translation Model

Compare and Migrate from PaLM to Gemini Model

Evaluating Multimodal Tasks

Evaluate LangChain

Compare Generative AI Models

Evaluate Generated Answers from RAG for QA with Gen AI Evaluation Service SDK

Customize Model-based Metrics to Evaluate a Gen AI Model

Evaluate Generative Model Tool Use

Get Started with Generative AI Evaluation Service

Gen AI and LLM Security - ReAct and RAG Attacks & Mitigations

Responsible AI with Gemini API in Vertex AI: Safety Ratings and Thresholds

Monitor Batch Prediction with Gemini API

Intro to Batch Predictions with the Gemini API using BigQuery Input

Imagen 3 Customized Images

Create High Quality Visual Assets with Imagen and Gemini

Create a Photoshop Document with Image Segmentation on Vertex AI

Enhance Imagen Prompts with Gemini

Image Segmentation on Vertex AI

Vertex AI Search with Filters & Metadata

Vertex AI Search - Querying Blended Data Apps and Summarization with Gemini

Create a Vertex AI Search Datastore and Search Engine

Building Search Applications with Vertex AI Search

Question Answering Over Documents

Bulk Question Answering with Vertex AI Search

Custom Embeddings with Vertex AI Search

Get Started with Chirp 2 - Advanced Features

Multimodal Function Calling with Claude Models

Vertex AI Search and Conversation Data Store Status Checker

Building a Research Multi Agent System - a Design Pattern Overview with Gemini 2.0

Intro to Generating and Executing Python Code with Gemini 2.0

Evaluating Agents - Evaluate a CrewAI agent with Vertex AI Gen AI Evaluation

Evaluating Agents - Evaluate a LangGraph agent with Vertex AI Gen AI Evaluation

Interactive Loan Application Assistant (Financial Services)

Vertex AI RAG Engine with Vertex AI Search

Evaluate a CrewAI agent on Vertex AI Reasoning Engine (Customized template)

Evaluating a LangChain Agent on Vertex AI Reasoning Engine (Prebuilt template)

Evaluate a LangGraph agent on Vertex AI Reasoning Engine (Customized template)

Supervised Fine Tuning with Gemini 1.5 Flash for change detection using the Google Gen AI SDK

Supervised fine-tuning with Gemini 1.5 Flash for Q&A using the Google Gen AI SDK