Imagen on Vertex AI Jupyter Notebook tutorials and samples

This document contains a list of Imagen on Vertex AI Jupyter Notebook tutorials and articles with code samples.

There are many environments in which you can host Jupyter Notebooks. You can:

  • Download them from GitHub and run them on your local machine
  • Download them from GitHub and run them on a Jupyter or JupyterLab server in your local network

  • Run them in the cloud using a service like Colaboratory (Colab) or Vertex AI Workbench.

Colab

Running a Jupyter Notebook in Colab is an easy way to get started quickly.

To open a notebook tutorial in Colab, click the Colab link in the notebook list. Colab creates a VM instance with all needed dependencies, launches the Colab environment, and loads the notebook.

Vertex AI Workbench

You can also run the notebook using user-managed notebooks. When you create a user-managed notebooks instance with Vertex AI Workbench, you have full control over the hosting VM. You can specify the configuration and environment of the hosting VM.

To open a notebook tutorial in a Vertex AI Workbench instance:

  1. Click the Vertex AI Workbench link in the notebook list. The link opens the Vertex AI Workbench console.
  2. In the Deploy to notebook screen, type a name for your new Vertex AI Workbench instance and click Create.
  3. In the Ready to open notebook dialog that appears after the instance starts, click Open.
  4. On the Confirm deployment to notebook server page, select Confirm.
  5. Before running the notebook, select Kernel > Restart Kernel and Clear all Outputs.

List of notebooks

Features Description Open in
Image generation (Imagen 2)
 Image Generation with Imagen on Vertex AI.
In this notebook, you explore the image generation features of Imagen using the Vertex AI SDK for Python. Learn more about Imagen's image generation feature.

Tutorial steps

  • Generate images using text prompts.
  • Experiment with different parameters, such as:
    • Increasing the number of images to be generated.
    • Fixing a seed number for reproducibility.
    • Influencing the output images using negative prompts.
 		Colab
GitHub
Vertex AI Workbench
Image generation (Imagen 2)

Text generation (Gemini 1.0 Pro)

Output text formatting (Gemini 1.0 Pro) 		Create high quality visual assets with Imagen and Gemini 1.0 Pro.
In this notebook, you create high quality visual assets for a restaurant menu using Imagen and Gemini 1.0 Pro. Learn more about image generation and multimodal models.

Tutorial steps

  • Generate an image prompt with Gemini 1.0 Pro.
  • Use Imagen to create high quality images using prompts.
  • Implement a short pipeline to produce highly-detailed visual assets.
 		Colab
GitHub
Vertex AI Workbench
Image editing (Imagen 2) Create high quality visual assets with Imagen 2 edit using automatically generated mask areas.
In this notebook, you will be exploring the image editing features of Imagen using the Vertex AI SDK for Python.

Tutorial steps

  • Edit an entire uploaded or generated image with a text prompt.
  • Define specific objects in an image to edit.
  • Edit the background of an image.
  • Edit the foreground of an image.
  • Remove the background or foreground of an image.
  • Experiment with different parameters, such as:
    • Reducing the dilation of a mask for thin objects.
    • Influencing the edited output image using negative prompt.
 		Colab
GitHub
Vertex AI Workbench
Image descriptions / visual captioning (Imagen) Visual captioning with Imagen on Vertex AI.
In this notebook, you will learn how to use the Vertex AI SDK for Python to generate visual captions for an image. Learn more about Imagen's image captioning feature.

Tutorial steps

  • Generate image captions using Imagen's visual captioning features.
  • Experiment with different parameters, such as:
    • Number of captions to be generated.
    • Language of the generated captions.
    • Type and version of model that is used to generate the captions.
 		Colab
GitHub
Vertex AI Workbench
Visual Question Answering (VQA) (Imagen) Visual Question Answering (VQA) with Imagen on Vertex AI.
In this notebook, you will learn how to use the Vertex AI SDK for Python to generate answers for questions you ask about an image. Learn more about Imagen's Visual Question Answering (VQA) feature.

Tutorial steps

  • Answer questions about images using Imagen's visual question answering feature.
  • Experiment with different parameters, such as:
    • Number of answers to be provided by the model.
 		Colab
GitHub
Vertex AI Workbench

List of articles with samples

Features Description Links
Image editing (Imagen)
 Background change with Imagen on Vertex AI: A step-by-step guide.
This article shows how to do mask-based editing using the Vertex AI SDK for Python. Changing the background involves the following steps:
  • Remove the existing background.
  • Create a mask and inverted mask image.
  • Encode an image to a string.
  • Create a request payload.
Learn more about Imagen's image editing feature. 		Article link
GitHub
Image generation (Imagen)

Text generation (PaLM 2 for Text) 		Google Imagen (through Google Cloud Vertex AI Studio) as fashion design assistant.

In this article, we will explore how generative AI can assist fashion designers in generating new ideas and designs using Google's suite of generative models for text and image generation. This article shows you how to use PaLM 2 for Text's text-bison model in the Google Cloud console and with the Vertex AI SDK for Python. It then shows you how to use the generated prompts to serve as input for image generation using Imagen's imagegeneration model.

Learn more about Imagen's image generation feature and PaLM 2 for Text's text generation feature.

Article link
Image generation (Imagen 2)

Image editing (Imagen) 		Image generation with Imagen and LangChain4j (Java).

In this article, we will explore how you can generate and edit images with Imagen in LangChain4j.

Learn more about Imagen's image generation and image editing features.

Article link