Large language models (LLMs) are deep learning models trained on massive amounts of text data. LLMs can translate language, summarize text, recognize objects and text in images, and complement search engines and recommendation systems.
Google provides the Gemini family of generative AI models designed for multimodal use cases; capable of processing information from multiple modalities, including images, videos, and text.
To see more learning resources, browse the Generative AI GitHub repo. Google data scientists, developers, and developer advocates manage this content.
Get started
Here are some notebooks, tutorials, and other examples to help you get started. Vertex AI offers Google Cloud console tutorials and Jupyter notebook tutorials that use the Vertex AI SDK for Python. You can open a notebook tutorial in Colab or download the notebook to your preferred environment.
Get started with Gemini
|
The Gemini model is a groundbreaking multimodal language model developed by Google AI, capable of extracting meaningful insights from a diverse array of data formats, including images, and video. This notebook explores various use cases with multimodal prompts.
Jupyter notebook: You can run this tutorial as a Jupyter notebook. |
Get started with Vertex AI Vertex AI Studio
|
Use Vertex AI Studio through the Google Cloud console without the need for the API or the Vertex AI SDK for Python. |
Best practices for prompt design
|
Learn how to design prompts to improve the quality of your responses from the model. This tutorial covers the essentials of prompt engineering, including some best practices.
Jupyter notebook: You can run this tutorial as a Jupyter notebook. |
LangChain 🦜️🔗
LangChain is a framework for developing applications powered by LLMs like the PaLM models. Use LangChain to bring external data, such as your files, other applications, and API data, to your LLMs.
To learn more about LangChain and how it works with Vertex AI, see the official LangChain and Vertex AI documentation.
Multimodal Retrieval Augmented Generation (RAG) with Gemini, Vector Search, and LangChain
|
Retrieval augmented generation (RAG) has become a popular paradigm for enabling LLMs to access external data and also as a mechanism for grounding to mitigate against hallucinations. Learn how to perform multimodal RAG where you will perform Q&A over a financial document filled with both text and images.
Jupyter notebook: You can run this tutorial as a Jupyter notebook. |
Augment Q&A Generation using LangChain 🦜️🔗 and BigQuery Vector Search
|
|
Learn how to implement a Question Answering (QA) system to improve an LLM's response by augmenting the LLM's knowledge with external data sources such as documents. This example uses Vertex AI Gemini 1.0 Pro for Text, Embeddings for Text API, BigQuery Vector Search, and LangChain
Jupyter notebook: You can run this tutorial as a Jupyter notebook. |
What's next
- Explore more resources in the Generative AI GitHub repo.
- See other Vertex AI notebook tutorials in the Tutorials overview.