Image understanding

Console

1. In the Vertex AI section of the Google Cloud console, go to the Vertex AI Studio page.

   Go to Vertex AI Studio

2. Click Open freeform.

3. Optional: Configure the model and parameters:

   • Model: Select a model.
   • Region: Select the region that you want to use.

   • Temperature: Use the slider or textbox to enter a value for temperature.

     The temperature is used for sampling during response generation, which occurs when topP and topK are applied. Temperature controls the degree of randomness in token selection. Lower temperatures are good for prompts that require a less open-ended or creative response, while higher temperatures can lead to more diverse or creative results. A temperature of 0 means that the highest probability tokens are always selected. In this case, responses for a given prompt are mostly deterministic, but a small amount of variation is still possible.

     If the model returns a response that's too generic, too short, or the model gives a fallback response, try increasing the temperature.

   • Output token limit: Use the slider or textbox to enter a value for the max output limit.

     Maximum number of tokens that can be generated in the response. A token is approximately four characters. 100 tokens correspond to roughly 60-80 words.

     Specify a lower value for shorter responses and a higher value for potentially longer responses.

   • Add stop sequence: Optional. Enter a stop sequence, which is a series of characters that includes spaces. If the model encounters a stop sequence, the response generation stops. The stop sequence isn't included in the response, and you can add up to five stop sequences.

4. Optional: To configure advanced parameters, click Advanced and configure as follows:

   Click to expand advanced configurations

   • Top-K: Use the slider or textbox to enter a value for top-K. (not supported for Gemini 1.5).

     Top-K changes how the model selects tokens for output. A top-K of 1 means the next selected token is the most probable among all tokens in the model's vocabulary (also called greedy decoding), while a top-K of 3 means that the next token is selected from among the three most probable tokens by using temperature.

     For each token selection step, the top-K tokens with the highest probabilities are sampled. Then tokens are further filtered based on top-P with the final token selected using temperature sampling.

     Specify a lower value for less random responses and a higher value for more random responses.

   • Top-P: Use the slider or textbox to enter a value for top-P. Tokens are selected from most probable to the least until the sum of their probabilities equals the value of top-P. For the least variable results, set top-P to 0.
   • Max responses: Use the slider or textbox to enter a value for the number of responses to generate.
   • Streaming responses: Enable to print responses as they're generated.
   • Safety filter threshold: Select the threshold of how likely you are to see responses that could be harmful.
   • Enable Grounding: Grounding isn't supported for multimodal prompts.

5. Click Insert Media, and select a source for your file.

   Upload

   Select the file that you want to upload and click Open.

   By URL

   Enter the URL of the file that you want to use and click Insert.

   Cloud Storage

   Select the bucket and then the file from the bucket that you want to import and click Select.

   Google Drive

   1. Choose an account and give consent to Vertex AI Studio to access your account the first time you select this option. You can upload multiple files that have a total size of up to 10 MB. A single file can't exceed 7 MB.
   2. Click the file that you want to add.

   3. Click Select.

      The file thumbnail displays in the Prompt pane. The total number of tokens also displays. If your prompt data exceeds the token limit, the tokens are truncated and aren't included in processing your data.

6. Enter your text prompt in the Prompt pane.

7. Optional: To view the Token ID to text and Token IDs, click the tokens count in the Prompt pane.

8. Click Submit.

9. Optional: To save your prompt to My prompts, click save_alt Save.

10. Optional: To get the Python code or a curl command for your prompt, click code Get code.

Python

Install

pip install --upgrade google-genai

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=global
export GOOGLE_GENAI_USE_VERTEXAI=True

from google import genai
from google.genai.types import HttpOptions, Part

client = genai.Client(http_options=HttpOptions(api_version="v1"))
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=[
        "What is shown in this image?",
        Part.from_uri(
            file_uri="gs://cloud-samples-data/generative-ai/image/scones.jpg",
            mime_type="image/jpeg",
        ),
    ],
)
print(response.text)
# Example response:
# The image shows a flat lay of blueberry scones arranged on parchment paper. There are ...

Go

Learn how to install or update the Go.

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=global
export GOOGLE_GENAI_USE_VERTEXAI=True

import (
	"context"
	"fmt"
	"io"

	genai "google.golang.org/genai"
)

// generateWithTextImage shows how to generate text using both text and image input
func generateWithTextImage(w io.Writer) error {
	ctx := context.Background()

	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		HTTPOptions: genai.HTTPOptions{APIVersion: "v1"},
	})
	if err != nil {
		return fmt.Errorf("failed to create genai client: %w", err)
	}

	modelName := "gemini-2.5-flash"
	contents := []*genai.Content{
		{Parts: []*genai.Part{
			{Text: "What is shown in this image?"},
			{FileData: &genai.FileData{
				// Image source: https://storage.googleapis.com/cloud-samples-data/generative-ai/image/scones.jpg
				FileURI:  "gs://cloud-samples-data/generative-ai/image/scones.jpg",
				MIMEType: "image/jpeg",
			}},
		},
			Role: "user"},
	}

	resp, err := client.Models.GenerateContent(ctx, modelName, contents, nil)
	if err != nil {
		return fmt.Errorf("failed to generate content: %w", err)
	}

	respText := resp.Text()

	fmt.Fprintln(w, respText)

	// Example response:
	// The image shows an overhead shot of a rustic, artistic arrangement on a surface that ...

	return nil
}

Node.js

Install

npm install @google/genai

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=global
export GOOGLE_GENAI_USE_VERTEXAI=True

const {GoogleGenAI} = require('@google/genai');

const GOOGLE_CLOUD_PROJECT = process.env.GOOGLE_CLOUD_PROJECT;
const GOOGLE_CLOUD_LOCATION = process.env.GOOGLE_CLOUD_LOCATION || 'global';

async function generateContent(
  projectId = GOOGLE_CLOUD_PROJECT,
  location = GOOGLE_CLOUD_LOCATION
) {
  const ai = new GoogleGenAI({
    vertexai: true,
    project: projectId,
    location: location,
  });

  const image = {
    fileData: {
      fileUri: 'gs://cloud-samples-data/generative-ai/image/scones.jpg',
      mimeType: 'image/jpeg',
    },
  };

  const response = await ai.models.generateContent({
    model: 'gemini-2.5-flash',
    contents: [image, 'What is shown in this image?'],
  });

  console.log(response.text);

  return response.text;
}

Java

Learn how to install or update the Java.

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=global
export GOOGLE_GENAI_USE_VERTEXAI=True

import com.google.genai.Client;
import com.google.genai.types.Content;
import com.google.genai.types.GenerateContentResponse;
import com.google.genai.types.HttpOptions;
import com.google.genai.types.Part;

public class TextGenerationWithTextAndImage {

  public static void main(String[] args) {
    // TODO(developer): Replace these variables before running the sample.
    String modelId = "gemini-2.5-flash";
    generateContent(modelId);
  }

  // Generates text with text and image input
  public static String generateContent(String modelId) {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (Client client =
        Client.builder()
            .location("global")
            .vertexAI(true)
            .httpOptions(HttpOptions.builder().apiVersion("v1").build())
            .build()) {

      GenerateContentResponse response =
          client.models.generateContent(
              modelId,
              Content.fromParts(
                  Part.fromText("What is shown in this image?"),
                  Part.fromUri(
                      "gs://cloud-samples-data/generative-ai/image/scones.jpg", "image/jpeg")),
              null);

      System.out.print(response.text());
      // Example response:
      // The image shows a flat lay of blueberry scones arranged on parchment paper. There are ...
      return response.text();
    }
  }
}

REST

After you set up your environment, you can use REST to test a text prompt. The following sample sends a request to the publisher model endpoint.

cat > request.json << 'EOF'
{
  "contents": {
    "role": "USER",
    "parts": [
      {
        "fileData": {
          "fileUri": "FILE_URI",
          "mimeType": "MIME_TYPE"
        }
      },
      {
        "text": "TEXT"
      }
    ]
  }
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/global/publishers/google/models/gemini-1.5-flash:generateContent"

@'
{
  "contents": {
    "role": "USER",
    "parts": [
      {
        "fileData": {
          "fileUri": "FILE_URI",
          "mimeType": "MIME_TYPE"
        }
      },
      {
        "text": "TEXT"
      }
    ]
  }
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/global/publishers/google/models/gemini-1.5-flash:generateContent" | Select-Object -Expand Content

{
  "candidates": [
    {
      "content": {
        "role": "model",
        "parts": [
          {
            "text": " The image shows a table with a cup of coffee, a bowl of blueberries, and a plate of scones with blueberries on it. There are also pink flowers on the table."
          }
        ]
      },
      "finishReason": "STOP",
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_HATE_SPEECH",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.027742893,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.07276838
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.026155617,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.07172113
        },
        {
          "category": "HARM_CATEGORY_HARASSMENT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.04304285,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.037608635
        },
        {
          "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.08803312,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.09203286
        }
      ]
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 265,
    "candidatesTokenCount": 35,
    "totalTokenCount": 300
  }
}

cat > request.json << 'EOF'
{
  "contents": {
    "role": "USER",
    "parts": [
      {
        "inlineData": {
          "data": "B64_BASE_IMAGE",
          "mimeType": "MIME_TYPE"
        }
      },
      {
        "text": "TEXT"
      }
    ]
  }
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/global/publishers/google/models/gemini-1.5-flash:generateContent"

@'
{
  "contents": {
    "role": "USER",
    "parts": [
      {
        "inlineData": {
          "data": "B64_BASE_IMAGE",
          "mimeType": "MIME_TYPE"
        }
      },
      {
        "text": "TEXT"
      }
    ]
  }
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/global/publishers/google/models/gemini-1.5-flash:generateContent" | Select-Object -Expand Content

{
  "candidates": [
    {
      "content": {
        "role": "model",
        "parts": [
          {
            "text": " The image shows a table with a cup of coffee, a bowl of blueberries, and a plate of scones with blueberries on it. There are also pink flowers on the table."
          }
        ]
      },
      "finishReason": "STOP",
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_HATE_SPEECH",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.027742893,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.07276838
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.026155617,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.07172113
        },
        {
          "category": "HARM_CATEGORY_HARASSMENT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.04304285,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.037608635
        },
        {
          "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.08803312,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.09203286
        }
      ]
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 265,
    "candidatesTokenCount": 35,
    "totalTokenCount": 300
  }
}

• Use the generateContent method to request that the response is returned after it's fully generated. To reduce the perception of latency to a human audience, stream the response as it's being generated by using the streamGenerateContent method.
• The multimodal model ID is located at the end of the URL before the method (for example, gemini-2.0-flash). This sample might support other models as well.

Console

1. In the Vertex AI section of the Google Cloud console, go to the Vertex AI Studio page.

   Go to Vertex AI Studio

2. Click Open freeform.

3. Optional: Configure the model and parameters:

   • Model: Select a model.
   • Region: Select the region that you want to use.

   • Temperature: Use the slider or textbox to enter a value for temperature.

     The temperature is used for sampling during response generation, which occurs when topP and topK are applied. Temperature controls the degree of randomness in token selection. Lower temperatures are good for prompts that require a less open-ended or creative response, while higher temperatures can lead to more diverse or creative results. A temperature of 0 means that the highest probability tokens are always selected. In this case, responses for a given prompt are mostly deterministic, but a small amount of variation is still possible.

     If the model returns a response that's too generic, too short, or the model gives a fallback response, try increasing the temperature.

   • Output token limit: Use the slider or textbox to enter a value for the max output limit.

     Maximum number of tokens that can be generated in the response. A token is approximately four characters. 100 tokens correspond to roughly 60-80 words.

     Specify a lower value for shorter responses and a higher value for potentially longer responses.

   • Add stop sequence: Optional. Enter a stop sequence, which is a series of characters that includes spaces. If the model encounters a stop sequence, the response generation stops. The stop sequence isn't included in the response, and you can add up to five stop sequences.

4. Optional: To configure advanced parameters, click Advanced and configure as follows:

   Click to expand advanced configurations

   • Top-K: Use the slider or textbox to enter a value for top-K. (not supported for Gemini 1.5).

     Top-K changes how the model selects tokens for output. A top-K of 1 means the next selected token is the most probable among all tokens in the model's vocabulary (also called greedy decoding), while a top-K of 3 means that the next token is selected from among the three most probable tokens by using temperature.

     For each token selection step, the top-K tokens with the highest probabilities are sampled. Then tokens are further filtered based on top-P with the final token selected using temperature sampling.

     Specify a lower value for less random responses and a higher value for more random responses.

   • Top-P: Use the slider or textbox to enter a value for top-P. Tokens are selected from most probable to the least until the sum of their probabilities equals the value of top-P. For the least variable results, set top-P to 0.
   • Max responses: Use the slider or textbox to enter a value for the number of responses to generate.
   • Streaming responses: Enable to print responses as they're generated.
   • Safety filter threshold: Select the threshold of how likely you are to see responses that could be harmful.
   • Enable Grounding: Grounding isn't supported for multimodal prompts.

5. Click Insert Media, and select a source for your file.

   Upload

   Select the file that you want to upload and click Open.

   By URL

   Enter the URL of the file that you want to use and click Insert.

   Cloud Storage

   Select the bucket and then the file from the bucket that you want to import and click Select.

   Google Drive

   1. Choose an account and give consent to Vertex AI Studio to access your account the first time you select this option. You can upload multiple files that have a total size of up to 10 MB. A single file can't exceed 7 MB.
   2. Click the file that you want to add.

   3. Click Select.

      The file thumbnail displays in the Prompt pane. The total number of tokens also displays. If your prompt data exceeds the token limit, the tokens are truncated and aren't included in processing your data.

6. Enter your text prompt in the Prompt pane.

7. Optional: To view the Token ID to text and Token IDs, click the tokens count in the Prompt pane.

8. Click Submit.

9. Optional: To save your prompt to My prompts, click save_alt Save.

10. Optional: To get the Python code or a curl command for your prompt, click code Get code.

Python

Install

pip install --upgrade google-genai

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=global
export GOOGLE_GENAI_USE_VERTEXAI=True

from google import genai
from google.genai.types import HttpOptions, Part

client = genai.Client(http_options=HttpOptions(api_version="v1"))

# Read content from GCS
gcs_file_img_path = "gs://cloud-samples-data/generative-ai/image/scones.jpg"

# Read content from a local file
with open("test_data/latte.jpg", "rb") as f:
    local_file_img_bytes = f.read()

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=[
        "Generate a list of all the objects contained in both images.",
        Part.from_uri(file_uri=gcs_file_img_path, mime_type="image/jpeg"),
        Part.from_bytes(data=local_file_img_bytes, mime_type="image/jpeg"),
    ],
)
print(response.text)
# Example response:
# Okay, here's the list of objects present in both images:
# ...

Go

Learn how to install or update the Go.

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=global
export GOOGLE_GENAI_USE_VERTEXAI=True

import (
	"context"
	"fmt"
	"io"
	"os"

	genai "google.golang.org/genai"
)

// generateWithMultiImg shows how to generate text using multiple image inputs.
func generateWithMultiImg(w io.Writer) error {
	ctx := context.Background()

	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		HTTPOptions: genai.HTTPOptions{APIVersion: "v1"},
	})
	if err != nil {
		return fmt.Errorf("failed to create genai client: %w", err)
	}

	// TODO(Developer): Update the path to file (image source:
	//   https://storage.googleapis.com/cloud-samples-data/generative-ai/image/latte.jpg )
	imageBytes, err := os.ReadFile("./latte.jpg")
	if err != nil {
		return fmt.Errorf("failed to read image: %w", err)
	}

	contents := []*genai.Content{
		{Parts: []*genai.Part{
			{Text: "Write an advertising jingle based on the items in both images."},
			{FileData: &genai.FileData{
				// Image source: https://storage.googleapis.com/cloud-samples-data/generative-ai/image/scones.jpg
				FileURI:  "gs://cloud-samples-data/generative-ai/image/scones.jpg",
				MIMEType: "image/jpeg",
			}},
			{InlineData: &genai.Blob{
				Data:     imageBytes,
				MIMEType: "image/jpeg",
			}},
		}},
	}
	modelName := "gemini-2.5-flash"

	resp, err := client.Models.GenerateContent(ctx, modelName, contents, nil)
	if err != nil {
		return fmt.Errorf("failed to generate content: %w", err)
	}

	respText := resp.Text()

	fmt.Fprintln(w, respText)

	// Example response:
	// Okay, here's an advertising jingle inspired by the blueberry scones, coffee, flowers, chocolate cake, and latte:
	//
	// (Upbeat, jazzy music)
	// ...

	return nil
}

Node.js

Install

npm install @google/genai

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=global
export GOOGLE_GENAI_USE_VERTEXAI=True

const {GoogleGenAI} = require('@google/genai');

const GOOGLE_CLOUD_PROJECT = process.env.GOOGLE_CLOUD_PROJECT;
const GOOGLE_CLOUD_LOCATION = process.env.GOOGLE_CLOUD_LOCATION || 'global';

async function generateContent(
  projectId = GOOGLE_CLOUD_PROJECT,
  location = GOOGLE_CLOUD_LOCATION
) {
  const ai = new GoogleGenAI({
    vertexai: true,
    project: projectId,
    location: location,
  });

  const image1 = {
    fileData: {
      fileUri: 'gs://cloud-samples-data/generative-ai/image/scones.jpg',
      mimeType: 'image/jpeg',
    },
  };

  const image2 = {
    fileData: {
      fileUri: 'gs://cloud-samples-data/generative-ai/image/fruit.png',
      mimeType: 'image/png',
    },
  };

  const response = await ai.models.generateContent({
    model: 'gemini-2.5-flash',
    contents: [
      image1,
      image2,
      'Generate a list of all the objects contained in both images.',
    ],
  });

  console.log(response.text);

  return response.text;
}

Java

Learn how to install or update the Java.

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=global
export GOOGLE_GENAI_USE_VERTEXAI=True

import com.google.genai.Client;
import com.google.genai.types.Content;
import com.google.genai.types.GenerateContentResponse;
import com.google.genai.types.HttpOptions;
import com.google.genai.types.Part;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

public class TextGenerationWithMultiImage {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String modelId = "gemini-2.5-flash";
    // Content from Google Cloud Storage
    String gcsFileImagePath = "gs://cloud-samples-data/generative-ai/image/scones.jpg";
    String localImageFilePath = "resources/latte.jpg";
    generateContent(modelId, gcsFileImagePath, localImageFilePath);
  }

  // Generates text with multiple images
  public static String generateContent(
      String modelId, String gcsFileImagePath, String localImageFilePath) throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (Client client =
        Client.builder()
            .location("global")
            .vertexAI(true)
            .httpOptions(HttpOptions.builder().apiVersion("v1").build())
            .build()) {

      // Read content from a local file.
      byte[] localFileImgBytes = Files.readAllBytes(Paths.get(localImageFilePath));

      GenerateContentResponse response =
          client.models.generateContent(
              modelId,
              Content.fromParts(
                  Part.fromText("Generate a list of all the objects contained in both images"),
                  Part.fromBytes(localFileImgBytes, "image/jpeg"),
                  Part.fromUri(gcsFileImagePath, "image/jpeg")),
              null);

      System.out.print(response.text());
      // Example response:
      // Okay, here's the list of objects present in both images:
      //
      // **Image 1 (Scones):**
      //
      // *   Scones
      // *   Plate
      // *   Jam/Preserve
      // *   Cream/Butter
      // *   Table/Surface
      // *   Napkin/Cloth (possibly)
      //
      // **Image 2 (Latte):**
      //
      // *   Latte/Coffee cup
      // *   Saucer
      // *   Spoon
      // *   Table/Surface
      // *   Foam/Latte art
      //
      // **Objects potentially in both (depending on interpretation and specific items):**
      //
      // *   Plate/Saucer (both are serving dishes)
      // *   Table/Surface
      return response.text();
    }
  }
}

REST

After you set up your environment, you can use REST to test a text prompt. The following sample sends a request to the publisher model endpoint.

Before using any of the request data, make the following replacements:

• PROJECT_ID: .
• FILE_URI1: The URI or URL of the file to include in the prompt. Acceptable values include the following:
  • Cloud Storage bucket URI: The object must either be publicly readable or reside in the same Google Cloud project that's sending the request. For gemini-2.0-flash and gemini-2.0-flash-lite, the size limit is 2 GB.
  • HTTP URL: The file URL must be publicly readable. You can specify one video file, one audio file, and up to 10 image files per request. Audio files, video files, and documents can't exceed 15 MB.
  • YouTube video URL:The YouTube video must be either owned by the account that you used to sign in to the Google Cloud console or is public. Only one YouTube video URL is supported per request.

  When specifying a fileURI, you must also specify the media type (mimeType) of the file. If VPC Service Controls is enabled, specifying a media file URL for fileURI is not supported.

  If you don't have an image file in Cloud Storage, then you can use the following publicly available file: gs://cloud-samples-data/vertex-ai/llm/prompts/landmark1.png with a mime type of image/png. To view this image, open the sample image file.

• MIME_TYPE: The media type of the file specified in the data or fileUri fields. Acceptable values include the following:

  Click to expand MIME types

  • application/pdf
  • audio/mpeg
  • audio/mp3
  • audio/wav
  • image/png
  • image/jpeg
  • image/webp
  • text/plain
  • video/mov
  • video/mpeg
  • video/mp4
  • video/mpg
  • video/avi
  • video/wmv
  • video/mpegps
  • video/flv
  For simplicity, this sample uses the same media type for all three input images.
• TEXT1: The text instructions to include in the prompt. For example, city: Rome, Landmark: the Colosseum
• FILE_URI2: The URI or URL of the file to include in the prompt. Acceptable values include the following:
  • Cloud Storage bucket URI: The object must either be publicly readable or reside in the same Google Cloud project that's sending the request. For gemini-2.0-flash and gemini-2.0-flash-lite, the size limit is 2 GB.
  • HTTP URL: The file URL must be publicly readable. You can specify one video file, one audio file, and up to 10 image files per request. Audio files, video files, and documents can't exceed 15 MB.
  • YouTube video URL:The YouTube video must be either owned by the account that you used to sign in to the Google Cloud console or is public. Only one YouTube video URL is supported per request.

  When specifying a fileURI, you must also specify the media type (mimeType) of the file. If VPC Service Controls is enabled, specifying a media file URL for fileURI is not supported.

  If you don't have an image file in Cloud Storage, then you can use the following publicly available file: gs://cloud-samples-data/vertex-ai/llm/prompts/landmark2.png with a mime type of image/png. To view this image, open the sample image file.

• TEXT2: The text instructions to include in the prompt. For example, city: Beijing, Landmark: Forbidden City
• FILE_URI3: The URI or URL of the file to include in the prompt. Acceptable values include the following:
  • Cloud Storage bucket URI: The object must either be publicly readable or reside in the same Google Cloud project that's sending the request. For gemini-2.0-flash and gemini-2.0-flash-lite, the size limit is 2 GB.
  • HTTP URL: The file URL must be publicly readable. You can specify one video file, one audio file, and up to 10 image files per request. Audio files, video files, and documents can't exceed 15 MB.
  • YouTube video URL:The YouTube video must be either owned by the account that you used to sign in to the Google Cloud console or is public. Only one YouTube video URL is supported per request.

  When specifying a fileURI, you must also specify the media type (mimeType) of the file. If VPC Service Controls is enabled, specifying a media file URL for fileURI is not supported.

  If you don't have an image file in Cloud Storage, then you can use the following publicly available file: gs://cloud-samples-data/vertex-ai/llm/prompts/landmark3.png with a mime type of image/png. To view this image, open the sample image file.

To send your request, choose one of these options:

cat > request.json << 'EOF'
{
  "contents": {
    "role": "USER",
    "parts": [
      {
        "fileData": {
          "fileUri": "FILE_URI1",
          "mimeType": "MIME_TYPE"
        }
      },
      {
        "text": "TEXT1"
      },
      {
        "fileData": {
          "fileUri": "FILE_URI2",
          "mimeType": "MIME_TYPE"
        }
      },
      {
        "text": "TEXT2"
      },
      {
        "fileData": {
          "fileUri": "FILE_URI3",
          "mimeType": "MIME_TYPE"
        }
      }
    ]
  }
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/global/publishers/google/models/gemini-2.5-flash:generateContent"

@'
{
  "contents": {
    "role": "USER",
    "parts": [
      {
        "fileData": {
          "fileUri": "FILE_URI1",
          "mimeType": "MIME_TYPE"
        }
      },
      {
        "text": "TEXT1"
      },
      {
        "fileData": {
          "fileUri": "FILE_URI2",
          "mimeType": "MIME_TYPE"
        }
      },
      {
        "text": "TEXT2"
      },
      {
        "fileData": {
          "fileUri": "FILE_URI3",
          "mimeType": "MIME_TYPE"
        }
      }
    ]
  }
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/global/publishers/google/models/gemini-2.5-flash:generateContent" | Select-Object -Expand Content

You should receive a JSON response similar to the following.

{
  "candidates": [
    {
      "content": {
        "role": "model",
        "parts": [
          {
            "text": "city: Rio de Janeiro, Landmark: Christ the Redeemer statue \n"
          }
        ]
      },
      "finishReason": "STOP",
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_HATE_SPEECH",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.05340333,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.08740791
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.13050689,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.10338596
        },
        {
          "category": "HARM_CATEGORY_HARASSMENT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.05399884,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.09947021
        },
        {
          "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.10576342,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.066934206
        }
      ]
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 791,
    "candidatesTokenCount": 14,
    "totalTokenCount": 805
  }
}

• Use the generateContent method to request that the response is returned after it's fully generated. To reduce the perception of latency to a human audience, stream the response as it's being generated by using the streamGenerateContent method.
• The multimodal model ID is located at the end of the URL before the method (for example, gemini-2.0-flash). This sample might support other models as well.