Package google.cloud.aiplatform.v1

Index

EvaluationService

Vertex AI Online Evaluation Service.

EvaluateInstances

rpc EvaluateInstances(EvaluateInstancesRequest) returns (EvaluateInstancesResponse)

Evaluates instances based on a given metric.

GenAiTuningService

A service for creating and managing GenAI Tuning Jobs.

CancelTuningJob

rpc CancelTuningJob(CancelTuningJobRequest) returns (Empty)

Cancels a TuningJob. Starts asynchronous cancellation on the TuningJob. The server makes a best effort to cancel the job, but success is not guaranteed. Clients can use GenAiTuningService.GetTuningJob or other methods to check whether the cancellation succeeded or whether the job completed despite cancellation. On successful cancellation, the TuningJob is not deleted; instead it becomes a job with a TuningJob.error value with a google.rpc.Status.code of 1, corresponding to Code.CANCELLED, and TuningJob.state is set to CANCELLED.

IAM Permissions

Requires the following IAM permission on the name resource:

  • aiplatform.tuningJobs.cancel

For more information, see the IAM documentation.
CreateTuningJob

rpc CreateTuningJob(CreateTuningJobRequest) returns (TuningJob)

Creates a TuningJob. A created TuningJob right away will be attempted to be run.

IAM Permissions

Requires the following IAM permission on the parent resource:

  • aiplatform.tuningJobs.create

For more information, see the IAM documentation.
GetTuningJob

rpc GetTuningJob(GetTuningJobRequest) returns (TuningJob)

Gets a TuningJob.

IAM Permissions

Requires the following IAM permission on the name resource:

  • aiplatform.tuningJobs.get

For more information, see the IAM documentation.
ListTuningJobs

rpc ListTuningJobs(ListTuningJobsRequest) returns (ListTuningJobsResponse)

Lists TuningJobs in a Location.

IAM Permissions

Requires the following IAM permission on the parent resource:

  • aiplatform.tuningJobs.list

For more information, see the IAM documentation.
RebaseTunedModel

rpc RebaseTunedModel(RebaseTunedModelRequest) returns (Operation)

Rebase a TunedModel.

IAM Permissions

Requires the following IAM permission on the parent resource:

  • aiplatform.tuningJobs.create

For more information, see the IAM documentation.

PredictionService

A service for online predictions and explanations.

ChatCompletions

rpc ChatCompletions(ChatCompletionsRequest) returns (HttpBody)

Exposes an OpenAI-compatible endpoint for chat completions.

IAM Permissions

Requires the following IAM permission on the endpoint resource:

  • aiplatform.endpoints.predict

For more information, see the IAM documentation.
GenerateContent

rpc GenerateContent(GenerateContentRequest) returns (GenerateContentResponse)

Generate content with multimodal inputs.

IAM Permissions

Requires the following IAM permission on the model resource:

  • aiplatform.endpoints.predict

For more information, see the IAM documentation.
Predict

rpc Predict(PredictRequest) returns (PredictResponse)

Perform an online prediction.

IAM Permissions

Requires the following IAM permission on the endpoint resource:

  • aiplatform.endpoints.predict

For more information, see the IAM documentation.
ServerStreamingPredict

rpc ServerStreamingPredict(StreamingPredictRequest) returns (StreamingPredictResponse)

Perform a server-side streaming online prediction request for Vertex LLM streaming.

IAM Permissions

Requires the following IAM permission on the endpoint resource:

  • aiplatform.endpoints.predict

For more information, see the IAM documentation.
StreamDirectPredict

rpc StreamDirectPredict(StreamDirectPredictRequest) returns (StreamDirectPredictResponse)

Perform a streaming online prediction request to a gRPC model server for Vertex first-party products and frameworks.

IAM Permissions

Requires the following IAM permission on the endpoint resource:

  • aiplatform.endpoints.predict

For more information, see the IAM documentation.
StreamDirectRawPredict

rpc StreamDirectRawPredict(StreamDirectRawPredictRequest) returns (StreamDirectRawPredictResponse)

Perform a streaming online prediction request to a gRPC model server for custom containers.

IAM Permissions

Requires the following IAM permission on the endpoint resource:

  • aiplatform.endpoints.predict

For more information, see the IAM documentation.
StreamGenerateContent

rpc StreamGenerateContent(GenerateContentRequest) returns (GenerateContentResponse)

Generate content with multimodal inputs with streaming support.

IAM Permissions

Requires the following IAM permission on the model resource:

  • aiplatform.endpoints.predict

For more information, see the IAM documentation.
StreamingPredict

rpc StreamingPredict(StreamingPredictRequest) returns (StreamingPredictResponse)

Perform a streaming online prediction request for Vertex first-party products and frameworks.

IAM Permissions

Requires the following IAM permission on the endpoint resource:

  • aiplatform.endpoints.predict

For more information, see the IAM documentation.
StreamingRawPredict

rpc StreamingRawPredict(StreamingRawPredictRequest) returns (StreamingRawPredictResponse)

Perform a streaming online prediction request through gRPC.

IAM Permissions

Requires the following IAM permission on the endpoint resource:

  • aiplatform.endpoints.predict

For more information, see the IAM documentation.

BleuInput

Input for bleu metric.

Fields
metric_spec BleuSpec

Required. Spec for bleu score metric.

instances[] BleuInstance

Required. Repeated bleu instances.

BleuInstance

Spec for bleu instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Required. Ground truth used to compare against the prediction.

BleuMetricValue

Bleu metric value for an instance.

Fields
score float

Output only. Bleu score.

BleuResults

Results for bleu metric.

Fields
bleu_metric_values[] BleuMetricValue

Output only. Bleu metric values.

BleuSpec

Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1.

Fields
use_effective_order bool

Optional. Whether to use_effective_order to compute bleu score.

Blob

Content blob.

It's preferred to send as text directly rather than raw bytes.

Fields
mime_type string

Required. The IANA standard MIME type of the source data.

data bytes

Required. Raw bytes.

CancelTuningJobRequest

Request message for GenAiTuningService.CancelTuningJob.

Fields
name string

Required. The name of the TuningJob to cancel. Format: projects/{project}/locations/{location}/tuningJobs/{tuning_job}

Candidate

A response candidate generated from the model.

Fields
index int32

Output only. Index of the candidate.

content Content

Output only. Content parts of the candidate.

avg_logprobs double

Output only. Average log probability score of the candidate.

logprobs_result LogprobsResult

Output only. Log-likelihood scores for the response tokens and top tokens

finish_reason FinishReason

Output only. The reason why the model stopped generating tokens. If empty, the model has not stopped generating the tokens.

safety_ratings[] SafetyRating

Output only. List of ratings for the safety of a response candidate.

There is at most one rating per category.

citation_metadata CitationMetadata

Output only. Source attribution of the generated content.

grounding_metadata GroundingMetadata

Output only. Metadata specifies sources used to ground generated content.

finish_message string

Output only. Describes the reason the mode stopped generating tokens in more detail. This is only filled when finish_reason is set.

FinishReason

The reason why the model stopped generating tokens. If empty, the model has not stopped generating the tokens.

Enums
FINISH_REASON_UNSPECIFIED The finish reason is unspecified.
STOP Token generation reached a natural stopping point or a configured stop sequence.
MAX_TOKENS Token generation reached the configured maximum output tokens.
SAFETY Token generation stopped because the content potentially contains safety violations. NOTE: When streaming, content is empty if content filters blocks the output.
RECITATION Token generation stopped because the content potentially contains copyright violations.
OTHER All other reasons that stopped the token generation.
BLOCKLIST Token generation stopped because the content contains forbidden terms.
PROHIBITED_CONTENT Token generation stopped for potentially containing prohibited content.
SPII Token generation stopped because the content potentially contains Sensitive Personally Identifiable Information (SPII).
MALFORMED_FUNCTION_CALL The function call generated by the model is invalid.

ChatCompletionsRequest

Request message for [PredictionService.ChatCompletions]

Fields
endpoint string

Required. The name of the endpoint requested to serve the prediction. Format: projects/{project}/locations/{location}/endpoints/{endpoint}

http_body HttpBody

Optional. The prediction input. Supports HTTP headers and arbitrary data payload.

Citation

Source attributions for content.

Fields
start_index int32

Output only. Start index into the content.

end_index int32

Output only. End index into the content.

uri string

Output only. Url reference of the attribution.

title string

Output only. Title of the attribution.

license string

Output only. License of the attribution.

publication_date Date

Output only. Publication date of the attribution.

CitationMetadata

A collection of source attributions for a piece of content.

Fields
citations[] Citation

Output only. List of citations.

CoherenceInput

Input for coherence metric.

Fields
metric_spec CoherenceSpec

Required. Spec for coherence score metric.

instance CoherenceInstance

Required. Coherence instance.

CoherenceInstance

Spec for coherence instance.

Fields
prediction string

Required. Output of the evaluated model.

CoherenceResult

Spec for coherence result.

Fields
explanation string

Output only. Explanation for coherence score.

score float

Output only. Coherence score.

confidence float

Output only. Confidence for coherence score.

CoherenceSpec

Spec for coherence score metric.

Fields
version int32

Optional. Which version to use for evaluation.

Content

The base structured datatype containing multi-part content of a message.

A Content includes a role field designating the producer of the Content and a parts field containing multi-part data that contains the content of the message turn.

Fields
role string

Optional. The producer of the content. Must be either 'user' or 'model'.

Useful to set for multi-turn conversations, otherwise can be left blank or unset.

parts[] Part

Required. Ordered Parts that constitute a single message. Parts may have different IANA MIME types.

CreateTuningJobRequest

Request message for GenAiTuningService.CreateTuningJob.

Fields
parent string

Required. The resource name of the Location to create the TuningJob in. Format: projects/{project}/locations/{location}

tuning_job TuningJob

Required. The TuningJob to create.

DynamicRetrievalConfig

Describes the options to customize dynamic retrieval.

Fields
mode Mode

The mode of the predictor to be used in dynamic retrieval.

dynamic_threshold float

Optional. The threshold to be used in dynamic retrieval. If not set, a system default value is used.

Mode

The mode of the predictor to be used in dynamic retrieval.

Enums
MODE_UNSPECIFIED Always trigger retrieval.
MODE_DYNAMIC Run retrieval only when system decides it is necessary.

EncryptionSpec

Represents a customer-managed encryption key spec that can be applied to a top-level resource.

Fields
kms_key_name string

Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

EvaluateInstancesRequest

Request message for EvaluationService.EvaluateInstances.

Fields
location string

Required. The resource name of the Location to evaluate the instances. Format: projects/{project}/locations/{location}

Union field metric_inputs. Instances and specs for evaluation metric_inputs can be only one of the following:
exact_match_input ExactMatchInput

Auto metric instances. Instances and metric spec for exact match metric.

bleu_input BleuInput

Instances and metric spec for bleu metric.

rouge_input RougeInput

Instances and metric spec for rouge metric.

fluency_input FluencyInput

LLM-based metric instance. General text generation metrics, applicable to other categories. Input for fluency metric.

coherence_input CoherenceInput

Input for coherence metric.

safety_input SafetyInput

Input for safety metric.

groundedness_input GroundednessInput

Input for groundedness metric.

fulfillment_input FulfillmentInput

Input for fulfillment metric.

summarization_quality_input SummarizationQualityInput

Input for summarization quality metric.

pairwise_summarization_quality_input PairwiseSummarizationQualityInput

Input for pairwise summarization quality metric.

summarization_helpfulness_input SummarizationHelpfulnessInput

Input for summarization helpfulness metric.

summarization_verbosity_input SummarizationVerbosityInput

Input for summarization verbosity metric.

question_answering_quality_input QuestionAnsweringQualityInput

Input for question answering quality metric.

pairwise_question_answering_quality_input PairwiseQuestionAnsweringQualityInput

Input for pairwise question answering quality metric.

question_answering_relevance_input QuestionAnsweringRelevanceInput

Input for question answering relevance metric.

question_answering_helpfulness_input QuestionAnsweringHelpfulnessInput

Input for question answering helpfulness metric.

question_answering_correctness_input QuestionAnsweringCorrectnessInput

Input for question answering correctness metric.

pointwise_metric_input PointwiseMetricInput

Input for pointwise metric.

pairwise_metric_input PairwiseMetricInput

Input for pairwise metric.

tool_call_valid_input ToolCallValidInput

Tool call metric instances. Input for tool call valid metric.

tool_name_match_input ToolNameMatchInput

Input for tool name match metric.

tool_parameter_key_match_input ToolParameterKeyMatchInput

Input for tool parameter key match metric.

tool_parameter_kv_match_input ToolParameterKVMatchInput

Input for tool parameter key value match metric.

EvaluateInstancesResponse

Response message for EvaluationService.EvaluateInstances.

Fields
Union field evaluation_results. Evaluation results will be served in the same order as presented in EvaluationRequest.instances. evaluation_results can be only one of the following:
exact_match_results ExactMatchResults

Auto metric evaluation results. Results for exact match metric.

bleu_results BleuResults

Results for bleu metric.

rouge_results RougeResults

Results for rouge metric.

fluency_result FluencyResult

LLM-based metric evaluation result. General text generation metrics, applicable to other categories. Result for fluency metric.

coherence_result CoherenceResult

Result for coherence metric.

safety_result SafetyResult

Result for safety metric.

groundedness_result GroundednessResult

Result for groundedness metric.

fulfillment_result FulfillmentResult

Result for fulfillment metric.

summarization_quality_result SummarizationQualityResult

Summarization only metrics. Result for summarization quality metric.

pairwise_summarization_quality_result PairwiseSummarizationQualityResult

Result for pairwise summarization quality metric.

summarization_helpfulness_result SummarizationHelpfulnessResult

Result for summarization helpfulness metric.

summarization_verbosity_result SummarizationVerbosityResult

Result for summarization verbosity metric.

question_answering_quality_result QuestionAnsweringQualityResult

Question answering only metrics. Result for question answering quality metric.

pairwise_question_answering_quality_result PairwiseQuestionAnsweringQualityResult

Result for pairwise question answering quality metric.

question_answering_relevance_result QuestionAnsweringRelevanceResult

Result for question answering relevance metric.

question_answering_helpfulness_result QuestionAnsweringHelpfulnessResult

Result for question answering helpfulness metric.

question_answering_correctness_result QuestionAnsweringCorrectnessResult

Result for question answering correctness metric.

pointwise_metric_result PointwiseMetricResult

Generic metrics. Result for pointwise metric.

pairwise_metric_result PairwiseMetricResult

Result for pairwise metric.

tool_call_valid_results ToolCallValidResults

Tool call metrics. Results for tool call valid metric.

tool_name_match_results ToolNameMatchResults

Results for tool name match metric.

tool_parameter_key_match_results ToolParameterKeyMatchResults

Results for tool parameter key match metric.

tool_parameter_kv_match_results ToolParameterKVMatchResults

Results for tool parameter key value match metric.

ExactMatchInput

Input for exact match metric.

Fields
metric_spec ExactMatchSpec

Required. Spec for exact match metric.

instances[] ExactMatchInstance

Required. Repeated exact match instances.

ExactMatchInstance

Spec for exact match instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Required. Ground truth used to compare against the prediction.

ExactMatchMetricValue

Exact match metric value for an instance.

Fields
score float

Output only. Exact match score.

ExactMatchResults

Results for exact match metric.

Fields
exact_match_metric_values[] ExactMatchMetricValue

Output only. Exact match metric values.

ExactMatchSpec

This type has no fields.

Spec for exact match metric - returns 1 if prediction and reference exactly matches, otherwise 0.

FileData

URI based data.

Fields
mime_type string

Required. The IANA standard MIME type of the source data.

file_uri string

Required. URI.

FluencyInput

Input for fluency metric.

Fields
metric_spec FluencySpec

Required. Spec for fluency score metric.

instance FluencyInstance

Required. Fluency instance.

FluencyInstance

Spec for fluency instance.

Fields
prediction string

Required. Output of the evaluated model.

FluencyResult

Spec for fluency result.

Fields
explanation string

Output only. Explanation for fluency score.

score float

Output only. Fluency score.

confidence float

Output only. Confidence for fluency score.

FluencySpec

Spec for fluency score metric.

Fields
version int32

Optional. Which version to use for evaluation.

FulfillmentInput

Input for fulfillment metric.

Fields
metric_spec FulfillmentSpec

Required. Spec for fulfillment score metric.

instance FulfillmentInstance

Required. Fulfillment instance.

FulfillmentInstance

Spec for fulfillment instance.

Fields
prediction string

Required. Output of the evaluated model.

instruction string

Required. Inference instruction prompt to compare prediction with.

FulfillmentResult

Spec for fulfillment result.

Fields
explanation string

Output only. Explanation for fulfillment score.

score float

Output only. Fulfillment score.

confidence float

Output only. Confidence for fulfillment score.

FulfillmentSpec

Spec for fulfillment metric.

Fields
version int32

Optional. Which version to use for evaluation.

FunctionCall

A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values.

Fields
name string

Required. The name of the function to call. Matches [FunctionDeclaration.name].

args Struct

Optional. Required. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.

FunctionCallingConfig

Function calling config.

Fields
mode Mode

Optional. Function calling mode.

allowed_function_names[] string

Optional. Function names to call. Only set when the Mode is ANY. Function names should match [FunctionDeclaration.name]. With mode set to ANY, model will predict a function call from the set of function names provided.

Mode

Function calling mode.

Enums
MODE_UNSPECIFIED Unspecified function calling mode. This value should not be used.
AUTO Default model behavior, model decides to predict either function calls or natural language response.
ANY Model is constrained to always predicting function calls only. If "allowed_function_names" are set, the predicted function calls will be limited to any one of "allowed_function_names", else the predicted function calls will be any one of the provided "function_declarations".
NONE Model will not predict any function calls. Model behavior is same as when not passing any function declarations.

FunctionDeclaration

Structured representation of a function declaration as defined by the OpenAPI 3.0 specification. Included in this declaration are the function name and parameters. This FunctionDeclaration is a representation of a block of code that can be used as a Tool by the model and executed by the client.

Fields
name string

Required. The name of the function to call. Must start with a letter or an underscore. Must be a-z, A-Z, 0-9, or contain underscores, dots and dashes, with a maximum length of 64.

description string

Optional. Description and purpose of the function. Model uses it to decide how and whether to call the function.

parameters Schema

Optional. Describes the parameters to this function in JSON Schema Object format. Reflects the Open API 3.03 Parameter Object. string Key: the name of the parameter. Parameter names are case sensitive. Schema Value: the Schema defining the type used for the parameter. For function with no parameters, this can be left unset. Parameter names must start with a letter or an underscore and must only contain chars a-z, A-Z, 0-9, or underscores with a maximum length of 64. Example with 1 required and 1 optional parameter: type: OBJECT properties: param1: type: STRING param2: type: INTEGER required: - param1

response Schema

Optional. Describes the output from this function in JSON Schema format. Reflects the Open API 3.03 Response Object. The Schema defines the type used for the response value of the function.

FunctionResponse

The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction.

Fields
name string

Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].

response Struct

Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.

GcsDestination

The Google Cloud Storage location where the output is to be written to.

Fields
output_uri_prefix string

Required. Google Cloud Storage URI to output directory. If the uri doesn't end with '/', a '/' will be automatically appended. The directory is created if it doesn't exist.

GenerateContentRequest

Request message for [PredictionService.GenerateContent].

Fields
model string

Required. The fully qualified name of the publisher model or tuned model endpoint to use.

Publisher model format: projects/{project}/locations/{location}/publishers/*/models/*

Tuned model endpoint format: projects/{project}/locations/{location}/endpoints/{endpoint}

contents[] Content

Required. The content of the current conversation with the model.

For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history + latest request.

tools[] Tool

Optional. A list of Tools the model may use to generate the next response.

A Tool is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model.

tool_config ToolConfig

Optional. Tool config. This config is shared for all tools provided in the request.

labels map<string, string>

Optional. The labels with user-defined metadata for the request. It is used for billing and reporting only.

Label keys and values can be no longer than 63 characters (Unicode codepoints) and can only contain lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter.

safety_settings[] SafetySetting

Optional. Per request settings for blocking unsafe content. Enforced on GenerateContentResponse.candidates.

generation_config GenerationConfig

Optional. Generation config.

system_instruction Content

Optional. The user provided system instructions for the model. Note: only text should be used in parts and content in each part will be in a separate paragraph.

GenerateContentResponse

Response message for [PredictionService.GenerateContent].

Fields
candidates[] Candidate

Output only. Generated candidates.

model_version string

Output only. The model version used to generate the response.

prompt_feedback PromptFeedback

Output only. Content filter results for a prompt sent in the request. Note: Sent only in the first stream chunk. Only happens when no candidates were generated due to content violations.

usage_metadata UsageMetadata

Usage metadata about the response(s).

PromptFeedback

Content filter results for a prompt sent in the request.

Fields
block_reason BlockedReason

Output only. Blocked reason.

safety_ratings[] SafetyRating

Output only. Safety ratings.

block_reason_message string

Output only. A readable block reason message.

BlockedReason

Blocked reason enumeration.

Enums
BLOCKED_REASON_UNSPECIFIED Unspecified blocked reason.
SAFETY Candidates blocked due to safety.
OTHER Candidates blocked due to other reason.
BLOCKLIST Candidates blocked due to the terms which are included from the terminology blocklist.
PROHIBITED_CONTENT Candidates blocked due to prohibited content.

UsageMetadata

Usage metadata about response(s).

Fields
prompt_token_count int32

Number of tokens in the request. When cached_content is set, this is still the total effective prompt size meaning this includes the number of tokens in the cached content.

candidates_token_count int32

Number of tokens in the response(s).

total_token_count int32

Total token count for prompt and response candidates.

GenerationConfig

Generation config.

Fields
stop_sequences[] string

Optional. Stop sequences.

response_mime_type string

Optional. Output response mimetype of the generated candidate text. Supported mimetype: - text/plain: (default) Text output. - application/json: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.

temperature float

Optional. Controls the randomness of predictions.

top_p float

Optional. If specified, nucleus sampling will be used.

top_k float

Optional. If specified, top-k sampling will be used.

candidate_count int32

Optional. Number of candidates to generate.

max_output_tokens int32

Optional. The maximum number of output tokens to generate per message.

response_logprobs bool

Optional. If true, export the logprobs results in response.

logprobs int32

Optional. Logit probabilities.

presence_penalty float

Optional. Positive penalties.

frequency_penalty float

Optional. Frequency penalties.

seed int32

Optional. Seed.

response_schema Schema

Optional. The Schema object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an OpenAPI 3.0 schema object. If set, a compatible response_mime_type must also be set. Compatible mimetypes: application/json: Schema for JSON response.

routing_config RoutingConfig

Optional. Routing configuration.

RoutingConfig

The configuration for routing the request to a specific model.

Fields
Union field routing_config. Routing mode. routing_config can be only one of the following:
auto_mode AutoRoutingMode

Automated routing.

manual_mode ManualRoutingMode

Manual routing.

AutoRoutingMode

When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference.

Fields
model_routing_preference ModelRoutingPreference

The model routing preference.

ModelRoutingPreference

The model routing preference.

Enums
UNKNOWN Unspecified model routing preference.
PRIORITIZE_QUALITY Prefer higher quality over low cost.
BALANCED Balanced model routing preference.
PRIORITIZE_COST Prefer lower cost over higher quality.

ManualRoutingMode

When manual routing is set, the specified model will be used directly.

Fields
model_name string

The model name to use. Only the public LLM models are accepted. e.g. 'gemini-1.5-pro-001'.

GenericOperationMetadata

Generic Metadata shared by all operations.

Fields
partial_failures[] Status

Output only. Partial failures encountered. E.g. single files that couldn't be read. This field should never exceed 20 entries. Status details field will contain standard Google Cloud error details.

create_time Timestamp

Output only. Time when the operation was created.

update_time Timestamp

Output only. Time when the operation was updated for the last time. If the operation has finished (successfully or not), this is the finish time.

GetTuningJobRequest

Request message for GenAiTuningService.GetTuningJob.

Fields
name string

Required. The name of the TuningJob resource. Format: projects/{project}/locations/{location}/tuningJobs/{tuning_job}

GoogleSearchRetrieval

Tool to retrieve public web data for grounding, powered by Google.

Fields
dynamic_retrieval_config DynamicRetrievalConfig

Specifies the dynamic retrieval configuration for the given source.

GroundednessInput

Input for groundedness metric.

Fields
metric_spec GroundednessSpec

Required. Spec for groundedness metric.

instance GroundednessInstance

Required. Groundedness instance.

GroundednessInstance

Spec for groundedness instance.

Fields
prediction string

Required. Output of the evaluated model.

context string

Required. Background information provided in context used to compare against the prediction.

GroundednessResult

Spec for groundedness result.

Fields
explanation string

Output only. Explanation for groundedness score.

score float

Output only. Groundedness score.

confidence float

Output only. Confidence for groundedness score.

GroundednessSpec

Spec for groundedness metric.

Fields
version int32

Optional. Which version to use for evaluation.

GroundingChunk

Grounding chunk.

Fields
Union field chunk_type. Chunk type. chunk_type can be only one of the following:
web Web

Grounding chunk from the web.

retrieved_context RetrievedContext

Grounding chunk from context retrieved by the retrieval tools.

RetrievedContext

Chunk from context retrieved by the retrieval tools.

Fields
uri string

URI reference of the attribution.

title string

Title of the attribution.

Web

Chunk from the web.

Fields
uri string

URI reference of the chunk.

title string

Title of the chunk.

GroundingMetadata

Metadata returned to client when grounding is enabled.

Fields
web_search_queries[] string

Optional. Web search queries for the following-up web search.

grounding_chunks[] GroundingChunk

List of supporting references retrieved from specified grounding source.

grounding_supports[] GroundingSupport

Optional. List of grounding support.

search_entry_point SearchEntryPoint

Optional. Google search entry for the following-up web searches.

retrieval_metadata RetrievalMetadata

Optional. Output only. Retrieval metadata.

GroundingSupport

Grounding support.

Fields
grounding_chunk_indices[] int32

A list of indices (into 'grounding_chunk') specifying the citations associated with the claim. For instance [1,3,4] means that grounding_chunk[1], grounding_chunk[3], grounding_chunk[4] are the retrieved content attributed to the claim.

confidence_scores[] float

Confidence score of the support references. Ranges from 0 to 1. 1 is the most confident. This list must have the same size as the grounding_chunk_indices.

segment Segment

Segment of the content this support belongs to.

HarmCategory

Harm categories that will block the content.

Enums
HARM_CATEGORY_UNSPECIFIED The harm category is unspecified.
HARM_CATEGORY_HATE_SPEECH The harm category is hate speech.
HARM_CATEGORY_DANGEROUS_CONTENT The harm category is dangerous content.
HARM_CATEGORY_HARASSMENT The harm category is harassment.
HARM_CATEGORY_SEXUALLY_EXPLICIT The harm category is sexually explicit content.
HARM_CATEGORY_CIVIC_INTEGRITY The harm category is civic integrity.

JobState

Describes the state of a job.

Enums
JOB_STATE_UNSPECIFIED The job state is unspecified.
JOB_STATE_QUEUED The job has been just created or resumed and processing has not yet begun.
JOB_STATE_PENDING The service is preparing to run the job.
JOB_STATE_RUNNING The job is in progress.
JOB_STATE_SUCCEEDED The job completed successfully.
JOB_STATE_FAILED The job failed.
JOB_STATE_CANCELLING The job is being cancelled. From this state the job may only go to either JOB_STATE_SUCCEEDED, JOB_STATE_FAILED or JOB_STATE_CANCELLED.
JOB_STATE_CANCELLED The job has been cancelled.
JOB_STATE_PAUSED The job has been stopped, and can be resumed.
JOB_STATE_EXPIRED The job has expired.
JOB_STATE_UPDATING The job is being updated. Only jobs in the RUNNING state can be updated. After updating, the job goes back to the RUNNING state.
JOB_STATE_PARTIALLY_SUCCEEDED The job is partially succeeded, some results may be missing due to errors.

ListTuningJobsRequest

Request message for GenAiTuningService.ListTuningJobs.

Fields
parent string

Required. The resource name of the Location to list the TuningJobs from. Format: projects/{project}/locations/{location}

filter string

Optional. The standard list filter.

page_size int32

Optional. The standard list page size.

page_token string

Optional. The standard list page token. Typically obtained via [ListTuningJob.next_page_token][] of the previous GenAiTuningService.ListTuningJob][] call.

ListTuningJobsResponse

Response message for GenAiTuningService.ListTuningJobs

Fields
tuning_jobs[] TuningJob

List of TuningJobs in the requested page.

next_page_token string

A token to retrieve the next page of results. Pass to ListTuningJobsRequest.page_token to obtain that page.

LogprobsResult

Logprobs Result

Fields
top_candidates[] TopCandidates

Length = total number of decoding steps.

chosen_candidates[] Candidate

Length = total number of decoding steps. The chosen candidates may or may not be in top_candidates.

Candidate

Candidate for the logprobs token and score.

Fields
token string

The candidate's token string value.

token_id int32

The candidate's token id value.

log_probability float

The candidate's log probability.

TopCandidates

Candidates with top log probabilities at each decoding step.

Fields
candidates[] Candidate

Sorted by log probability in descending order.

PairwiseChoice

Pairwise prediction autorater preference.

Enums
PAIRWISE_CHOICE_UNSPECIFIED Unspecified prediction choice.
BASELINE Baseline prediction wins
CANDIDATE Candidate prediction wins
TIE Winner cannot be determined

PairwiseMetricInput

Input for pairwise metric.

Fields
metric_spec PairwiseMetricSpec

Required. Spec for pairwise metric.

instance PairwiseMetricInstance

Required. Pairwise metric instance.

PairwiseMetricInstance

Pairwise metric instance. Usually one instance corresponds to one row in an evaluation dataset.

Fields
Union field instance. Instance for pairwise metric. instance can be only one of the following:
json_instance string

Instance specified as a json string. String key-value pairs are expected in the json_instance to render PairwiseMetricSpec.instance_prompt_template.

PairwiseMetricResult

Spec for pairwise metric result.

Fields
pairwise_choice PairwiseChoice

Output only. Pairwise metric choice.

explanation string

Output only. Explanation for pairwise metric score.

PairwiseMetricSpec

Spec for pairwise metric.

Fields
metric_prompt_template string

Required. Metric prompt template for pairwise metric.

PairwiseQuestionAnsweringQualityInput

Input for pairwise question answering quality metric.

Fields
metric_spec PairwiseQuestionAnsweringQualitySpec

Required. Spec for pairwise question answering quality score metric.

instance PairwiseQuestionAnsweringQualityInstance

Required. Pairwise question answering quality instance.

PairwiseQuestionAnsweringQualityInstance

Spec for pairwise question answering quality instance.

Fields
prediction string

Required. Output of the candidate model.

baseline_prediction string

Required. Output of the baseline model.

reference string

Optional. Ground truth used to compare against the prediction.

context string

Required. Text to answer the question.

instruction string

Required. Question Answering prompt for LLM.

PairwiseQuestionAnsweringQualityResult

Spec for pairwise question answering quality result.

Fields
pairwise_choice PairwiseChoice

Output only. Pairwise question answering prediction choice.

explanation string

Output only. Explanation for question answering quality score.

confidence float

Output only. Confidence for question answering quality score.

PairwiseQuestionAnsweringQualitySpec

Spec for pairwise question answering quality score metric.

Fields
use_reference bool

Optional. Whether to use instance.reference to compute question answering quality.

version int32

Optional. Which version to use for evaluation.

PairwiseSummarizationQualityInput

Input for pairwise summarization quality metric.

Fields
metric_spec PairwiseSummarizationQualitySpec

Required. Spec for pairwise summarization quality score metric.

instance PairwiseSummarizationQualityInstance

Required. Pairwise summarization quality instance.

PairwiseSummarizationQualityInstance

Spec for pairwise summarization quality instance.

Fields
prediction string

Required. Output of the candidate model.

baseline_prediction string

Required. Output of the baseline model.

reference string

Optional. Ground truth used to compare against the prediction.

context string

Required. Text to be summarized.

instruction string

Required. Summarization prompt for LLM.

PairwiseSummarizationQualityResult

Spec for pairwise summarization quality result.

Fields
pairwise_choice PairwiseChoice

Output only. Pairwise summarization prediction choice.

explanation string

Output only. Explanation for summarization quality score.

confidence float

Output only. Confidence for summarization quality score.

PairwiseSummarizationQualitySpec

Spec for pairwise summarization quality score metric.

Fields
use_reference bool

Optional. Whether to use instance.reference to compute pairwise summarization quality.

version int32

Optional. Which version to use for evaluation.

Part

A datatype containing media that is part of a multi-part Content message.

A Part consists of data which has an associated datatype. A Part can only contain one of the accepted types in Part.data.

A Part must have a fixed IANA MIME type identifying the type and subtype of the media if inline_data or file_data field is filled with raw bytes.

Fields

Union field data.

data can be only one of the following:

text string

Optional. Text part (can be code).

inline_data Blob

Optional. Inlined bytes data.

file_data FileData

Optional. URI based data.

function_call FunctionCall

Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.

function_response FunctionResponse

Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.

Union field metadata.

metadata can be only one of the following:

video_metadata VideoMetadata

Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.

PointwiseMetricInput

Input for pointwise metric.

Fields
metric_spec PointwiseMetricSpec

Required. Spec for pointwise metric.

instance PointwiseMetricInstance

Required. Pointwise metric instance.

PointwiseMetricInstance

Pointwise metric instance. Usually one instance corresponds to one row in an evaluation dataset.

Fields
Union field instance. Instance for pointwise metric. instance can be only one of the following:
json_instance string

Instance specified as a json string. String key-value pairs are expected in the json_instance to render PointwiseMetricSpec.instance_prompt_template.

PointwiseMetricResult

Spec for pointwise metric result.

Fields
explanation string

Output only. Explanation for pointwise metric score.

score float

Output only. Pointwise metric score.

PointwiseMetricSpec

Spec for pointwise metric.

Fields
metric_prompt_template string

Required. Metric prompt template for pointwise metric.

PredictRequest

Request message for PredictionService.Predict.

Fields
endpoint string

Required. The name of the Endpoint requested to serve the prediction. Format: projects/{project}/locations/{location}/endpoints/{endpoint}

instances[] Value

Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint's DeployedModels' Model's PredictSchemata's instance_schema_uri.

parameters Value

The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint's DeployedModels' Model's PredictSchemata's parameters_schema_uri.

PredictResponse

Response message for PredictionService.Predict.

Fields
predictions[] Value

The predictions that are the output of the predictions call. The schema of any single prediction may be specified via Endpoint's DeployedModels' Model's PredictSchemata's prediction_schema_uri.

deployed_model_id string

ID of the Endpoint's DeployedModel that served this prediction.

model string

Output only. The resource name of the Model which is deployed as the DeployedModel that this prediction hits.

model_version_id string

Output only. The version ID of the Model which is deployed as the DeployedModel that this prediction hits.

model_display_name string

Output only. The display name of the Model which is deployed as the DeployedModel that this prediction hits.

metadata Value

Output only. Request-level metadata returned by the model. The metadata type will be dependent upon the model implementation.

QuestionAnsweringCorrectnessInput

Input for question answering correctness metric.

Fields
metric_spec QuestionAnsweringCorrectnessSpec

Required. Spec for question answering correctness score metric.

instance QuestionAnsweringCorrectnessInstance

Required. Question answering correctness instance.

QuestionAnsweringCorrectnessInstance

Spec for question answering correctness instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Optional. Ground truth used to compare against the prediction.

context string

Optional. Text provided as context to answer the question.

instruction string

Required. The question asked and other instruction in the inference prompt.

QuestionAnsweringCorrectnessResult

Spec for question answering correctness result.

Fields
explanation string

Output only. Explanation for question answering correctness score.

score float

Output only. Question Answering Correctness score.

confidence float

Output only. Confidence for question answering correctness score.

QuestionAnsweringCorrectnessSpec

Spec for question answering correctness metric.

Fields
use_reference bool

Optional. Whether to use instance.reference to compute question answering correctness.

version int32

Optional. Which version to use for evaluation.

QuestionAnsweringHelpfulnessInput

Input for question answering helpfulness metric.

Fields
metric_spec QuestionAnsweringHelpfulnessSpec

Required. Spec for question answering helpfulness score metric.

instance QuestionAnsweringHelpfulnessInstance

Required. Question answering helpfulness instance.

QuestionAnsweringHelpfulnessInstance

Spec for question answering helpfulness instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Optional. Ground truth used to compare against the prediction.

context string

Optional. Text provided as context to answer the question.

instruction string

Required. The question asked and other instruction in the inference prompt.

QuestionAnsweringHelpfulnessResult

Spec for question answering helpfulness result.

Fields
explanation string

Output only. Explanation for question answering helpfulness score.

score float

Output only. Question Answering Helpfulness score.

confidence float

Output only. Confidence for question answering helpfulness score.

QuestionAnsweringHelpfulnessSpec

Spec for question answering helpfulness metric.

Fields
use_reference bool

Optional. Whether to use instance.reference to compute question answering helpfulness.

version int32

Optional. Which version to use for evaluation.

QuestionAnsweringQualityInput

Input for question answering quality metric.

Fields
metric_spec QuestionAnsweringQualitySpec

Required. Spec for question answering quality score metric.

instance QuestionAnsweringQualityInstance

Required. Question answering quality instance.

QuestionAnsweringQualityInstance

Spec for question answering quality instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Optional. Ground truth used to compare against the prediction.

context string

Required. Text to answer the question.

instruction string

Required. Question Answering prompt for LLM.

QuestionAnsweringQualityResult

Spec for question answering quality result.

Fields
explanation string

Output only. Explanation for question answering quality score.

score float

Output only. Question Answering Quality score.

confidence float

Output only. Confidence for question answering quality score.

QuestionAnsweringQualitySpec

Spec for question answering quality score metric.

Fields
use_reference bool

Optional. Whether to use instance.reference to compute question answering quality.

version int32

Optional. Which version to use for evaluation.

QuestionAnsweringRelevanceInput

Input for question answering relevance metric.

Fields
metric_spec QuestionAnsweringRelevanceSpec

Required. Spec for question answering relevance score metric.

instance QuestionAnsweringRelevanceInstance

Required. Question answering relevance instance.

QuestionAnsweringRelevanceInstance

Spec for question answering relevance instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Optional. Ground truth used to compare against the prediction.

context string

Optional. Text provided as context to answer the question.

instruction string

Required. The question asked and other instruction in the inference prompt.

QuestionAnsweringRelevanceResult

Spec for question answering relevance result.

Fields
explanation string

Output only. Explanation for question answering relevance score.

score float

Output only. Question Answering Relevance score.

confidence float

Output only. Confidence for question answering relevance score.

QuestionAnsweringRelevanceSpec

Spec for question answering relevance metric.

Fields
use_reference bool

Optional. Whether to use instance.reference to compute question answering relevance.

version int32

Optional. Which version to use for evaluation.

RebaseTunedModelOperationMetadata

Runtime operation information for GenAiTuningService.RebaseTunedModel.

Fields
generic_metadata GenericOperationMetadata

The common part of the operation generic information.

RebaseTunedModelRequest

Request message for GenAiTuningService.RebaseTunedModel.

Fields
parent string

Required. The resource name of the Location into which to rebase the Model. Format: projects/{project}/locations/{location}

tuned_model_ref TunedModelRef

Required. TunedModel reference to retrieve the legacy model information.

tuning_job TuningJob

Optional. The TuningJob to be updated. Users can use this TuningJob field to overwrite tuning configs.

artifact_destination GcsDestination

Optional. The Google Cloud Storage location to write the artifacts.

deploy_to_same_endpoint bool

Optional. By default, bison to gemini migration will always create new model/endpoint, but for gemini-1.0 to gemini-1.5 migration, we default deploy to the same endpoint. See details in this Section.

Retrieval

Defines a retrieval tool that model can call to access external knowledge.

Fields
disable_attribution
(deprecated) bool

Optional. Deprecated. This option is no longer supported.

Union field source. The source of the retrieval. source can be only one of the following:
vertex_rag_store VertexRagStore

Set to use data source powered by Vertex RAG store. User data is uploaded via the VertexRagDataService.

RetrievalMetadata

Metadata related to retrieval in the grounding flow.

Fields
google_search_dynamic_retrieval_score float

Optional. Score indicating how likely information from Google Search could help answer the prompt. The score is in the range [0, 1], where 0 is the least likely and 1 is the most likely. This score is only populated when Google Search grounding and dynamic retrieval is enabled. It will be compared to the threshold to determine whether to trigger Google Search.

RougeInput

Input for rouge metric.

Fields
metric_spec RougeSpec

Required. Spec for rouge score metric.

instances[] RougeInstance

Required. Repeated rouge instances.

RougeInstance

Spec for rouge instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Required. Ground truth used to compare against the prediction.

RougeMetricValue

Rouge metric value for an instance.

Fields
score float

Output only. Rouge score.

RougeResults

Results for rouge metric.

Fields
rouge_metric_values[] RougeMetricValue

Output only. Rouge metric values.

RougeSpec

Spec for rouge score metric - calculates the recall of n-grams in prediction as compared to reference - returns a score ranging between 0 and 1.

Fields
rouge_type string

Optional. Supported rouge types are rougen[1-9], rougeL, and rougeLsum.

use_stemmer bool

Optional. Whether to use stemmer to compute rouge score.

split_summaries bool

Optional. Whether to split summaries while using rougeLsum.

SafetyInput

Input for safety metric.

Fields
metric_spec SafetySpec

Required. Spec for safety metric.

instance SafetyInstance

Required. Safety instance.

SafetyInstance

Spec for safety instance.

Fields
prediction string

Required. Output of the evaluated model.

SafetyRating

Safety rating corresponding to the generated content.

Fields
category HarmCategory

Output only. Harm category.

probability HarmProbability

Output only. Harm probability levels in the content.

probability_score float

Output only. Harm probability score.

severity HarmSeverity

Output only. Harm severity levels in the content.

severity_score float

Output only. Harm severity score.

blocked bool

Output only. Indicates whether the content was filtered out because of this rating.

HarmProbability

Harm probability levels in the content.

Enums
HARM_PROBABILITY_UNSPECIFIED Harm probability unspecified.
NEGLIGIBLE Negligible level of harm.
LOW Low level of harm.
MEDIUM Medium level of harm.
HIGH High level of harm.

HarmSeverity

Harm severity levels.

Enums
HARM_SEVERITY_UNSPECIFIED Harm severity unspecified.
HARM_SEVERITY_NEGLIGIBLE Negligible level of harm severity.
HARM_SEVERITY_LOW Low level of harm severity.
HARM_SEVERITY_MEDIUM Medium level of harm severity.
HARM_SEVERITY_HIGH High level of harm severity.

SafetyResult

Spec for safety result.

Fields
explanation string

Output only. Explanation for safety score.

score float

Output only. Safety score.

confidence float

Output only. Confidence for safety score.

SafetySetting

Safety settings.

Fields
category HarmCategory

Required. Harm category.

threshold HarmBlockThreshold

Required. The harm block threshold.

method HarmBlockMethod

Optional. Specify if the threshold is used for probability or severity score. If not specified, the threshold is used for probability score.

HarmBlockMethod

Probability vs severity.

Enums
HARM_BLOCK_METHOD_UNSPECIFIED The harm block method is unspecified.
SEVERITY The harm block method uses both probability and severity scores.
PROBABILITY The harm block method uses the probability score.

HarmBlockThreshold

Probability based thresholds levels for blocking.

Enums
HARM_BLOCK_THRESHOLD_UNSPECIFIED Unspecified harm block threshold.
BLOCK_LOW_AND_ABOVE Block low threshold and above (i.e. block more).
BLOCK_MEDIUM_AND_ABOVE Block medium threshold and above.
BLOCK_ONLY_HIGH Block only high threshold (i.e. block less).
BLOCK_NONE Block none.
OFF Turn off the safety filter.

SafetySpec

Spec for safety metric.

Fields
version int32

Optional. Which version to use for evaluation.

Schema

Schema is used to define the format of input/output data. Represents a select subset of an OpenAPI 3.0 schema object. More fields may be added in the future as needed.

Fields
type Type

Optional. The type of the data.

format string

Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc

title string

Optional. The title of the Schema.

description string

Optional. The description of the data.

nullable bool

Optional. Indicates if the value may be null.

default Value

Optional. Default value of the data.

items Schema

Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.

min_items int64

Optional. Minimum number of the elements for Type.ARRAY.

max_items int64

Optional. Maximum number of the elements for Type.ARRAY.

enum[] string

Optional. Possible values of the element of Type.STRING with enum format. For example we can define an Enum Direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]}

properties map<string, Schema>

Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.

property_ordering[] string

Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.

required[] string

Optional. Required properties of Type.OBJECT.

min_properties int64

Optional. Minimum number of the properties for Type.OBJECT.

max_properties int64

Optional. Maximum number of the properties for Type.OBJECT.

minimum double

Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER

maximum double

Optional. Maximum value of the Type.INTEGER and Type.NUMBER

min_length int64

Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING

max_length int64

Optional. Maximum length of the Type.STRING

pattern string

Optional. Pattern of the Type.STRING to restrict a string to a regular expression.

example Value

Optional. Example of the object. Will only populated when the object is the root.

any_of[] Schema

Optional. The value should be validated against any (one or more) of the subschemas in the list.

SearchEntryPoint

Google search entry point.

Fields
rendered_content string

Optional. Web content snippet that can be embedded in a web page or an app webview.

sdk_blob bytes

Optional. Base64 encoded JSON representing array of <search term, search url> tuple.

Segment

Segment of the content.

Fields
part_index int32

Output only. The index of a Part object within its parent Content object.

start_index int32

Output only. Start index in the given Part, measured in bytes. Offset from the start of the Part, inclusive, starting at zero.

end_index int32

Output only. End index in the given Part, measured in bytes. Offset from the start of the Part, exclusive, starting at zero.

text string

Output only. The text corresponding to the segment from the response.

StreamDirectPredictRequest

Request message for PredictionService.StreamDirectPredict.

The first message must contain endpoint field and optionally [input][]. The subsequent messages must contain [input][].

Fields
endpoint string

Required. The name of the Endpoint requested to serve the prediction. Format: projects/{project}/locations/{location}/endpoints/{endpoint}

inputs[] Tensor

Optional. The prediction input.

parameters Tensor

Optional. The parameters that govern the prediction.

StreamDirectPredictResponse

Response message for PredictionService.StreamDirectPredict.

Fields
outputs[] Tensor

The prediction output.

parameters Tensor

The parameters that govern the prediction.

StreamDirectRawPredictRequest

Request message for PredictionService.StreamDirectRawPredict.

The first message must contain endpoint and method_name fields and optionally input. The subsequent messages must contain input. method_name in the subsequent messages have no effect.

Fields
endpoint string

Required. The name of the Endpoint requested to serve the prediction. Format: projects/{project}/locations/{location}/endpoints/{endpoint}

method_name string

Optional. Fully qualified name of the API method being invoked to perform predictions.

Format: /namespace.Service/Method/ Example: /tensorflow.serving.PredictionService/Predict

input bytes

Optional. The prediction input.

StreamDirectRawPredictResponse

Response message for PredictionService.StreamDirectRawPredict.

Fields
output bytes

The prediction output.

StreamingPredictRequest

Request message for PredictionService.StreamingPredict.

The first message must contain endpoint field and optionally [input][]. The subsequent messages must contain [input][].

Fields
endpoint string

Required. The name of the Endpoint requested to serve the prediction. Format: projects/{project}/locations/{location}/endpoints/{endpoint}

inputs[] Tensor

The prediction input.

parameters Tensor

The parameters that govern the prediction.

StreamingPredictResponse

Response message for PredictionService.StreamingPredict.

Fields
outputs[] Tensor

The prediction output.

parameters Tensor

The parameters that govern the prediction.

StreamingRawPredictRequest

Request message for PredictionService.StreamingRawPredict.

The first message must contain endpoint and method_name fields and optionally input. The subsequent messages must contain input. method_name in the subsequent messages have no effect.

Fields
endpoint string

Required. The name of the Endpoint requested to serve the prediction. Format: projects/{project}/locations/{location}/endpoints/{endpoint}

method_name string

Fully qualified name of the API method being invoked to perform predictions.

Format: /namespace.Service/Method/ Example: /tensorflow.serving.PredictionService/Predict

input bytes

The prediction input.

StreamingRawPredictResponse

Response message for PredictionService.StreamingRawPredict.

Fields
output bytes

The prediction output.

SummarizationHelpfulnessInput

Input for summarization helpfulness metric.

Fields
metric_spec SummarizationHelpfulnessSpec

Required. Spec for summarization helpfulness score metric.

instance SummarizationHelpfulnessInstance

Required. Summarization helpfulness instance.

SummarizationHelpfulnessInstance

Spec for summarization helpfulness instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Optional. Ground truth used to compare against the prediction.

context string

Required. Text to be summarized.

instruction string

Optional. Summarization prompt for LLM.

SummarizationHelpfulnessResult

Spec for summarization helpfulness result.

Fields
explanation string

Output only. Explanation for summarization helpfulness score.

score float

Output only. Summarization Helpfulness score.

confidence float

Output only. Confidence for summarization helpfulness score.

SummarizationHelpfulnessSpec

Spec for summarization helpfulness score metric.

Fields
use_reference bool

Optional. Whether to use instance.reference to compute summarization helpfulness.

version int32

Optional. Which version to use for evaluation.

SummarizationQualityInput

Input for summarization quality metric.

Fields
metric_spec SummarizationQualitySpec

Required. Spec for summarization quality score metric.

instance SummarizationQualityInstance

Required. Summarization quality instance.

SummarizationQualityInstance

Spec for summarization quality instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Optional. Ground truth used to compare against the prediction.

context string

Required. Text to be summarized.

instruction string

Required. Summarization prompt for LLM.

SummarizationQualityResult

Spec for summarization quality result.

Fields
explanation string

Output only. Explanation for summarization quality score.

score float

Output only. Summarization Quality score.

confidence float

Output only. Confidence for summarization quality score.

SummarizationQualitySpec

Spec for summarization quality score metric.

Fields
use_reference bool

Optional. Whether to use instance.reference to compute summarization quality.

version int32

Optional. Which version to use for evaluation.

SummarizationVerbosityInput

Input for summarization verbosity metric.

Fields
metric_spec SummarizationVerbositySpec

Required. Spec for summarization verbosity score metric.

instance SummarizationVerbosityInstance

Required. Summarization verbosity instance.

SummarizationVerbosityInstance

Spec for summarization verbosity instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Optional. Ground truth used to compare against the prediction.

context string

Required. Text to be summarized.

instruction string

Optional. Summarization prompt for LLM.

SummarizationVerbosityResult

Spec for summarization verbosity result.

Fields
explanation string

Output only. Explanation for summarization verbosity score.

score float

Output only. Summarization Verbosity score.

confidence float

Output only. Confidence for summarization verbosity score.

SummarizationVerbositySpec

Spec for summarization verbosity score metric.

Fields
use_reference bool

Optional. Whether to use instance.reference to compute summarization verbosity.

version int32

Optional. Which version to use for evaluation.

SupervisedHyperParameters

Hyperparameters for SFT.

Fields
epoch_count int64

Optional. Number of complete passes the model makes over the entire training dataset during training.

learning_rate_multiplier double

Optional. Multiplier for adjusting the default learning rate.

adapter_size AdapterSize

Optional. Adapter size for tuning.

AdapterSize

Supported adapter sizes for tuning.

Enums
ADAPTER_SIZE_UNSPECIFIED Adapter size is unspecified.
ADAPTER_SIZE_ONE Adapter size 1.
ADAPTER_SIZE_FOUR Adapter size 4.
ADAPTER_SIZE_EIGHT Adapter size 8.
ADAPTER_SIZE_SIXTEEN Adapter size 16.
ADAPTER_SIZE_THIRTY_TWO Adapter size 32.

SupervisedTuningDataStats

Tuning data statistics for Supervised Tuning.

Fields
tuning_dataset_example_count int64

Output only. Number of examples in the tuning dataset.

total_tuning_character_count int64

Output only. Number of tuning characters in the tuning dataset.

total_billable_character_count
(deprecated) int64

Output only. Number of billable characters in the tuning dataset.

total_billable_token_count int64

Output only. Number of billable tokens in the tuning dataset.

tuning_step_count int64

Output only. Number of tuning steps for this Tuning Job.

user_input_token_distribution SupervisedTuningDatasetDistribution

Output only. Dataset distributions for the user input tokens.

user_output_token_distribution SupervisedTuningDatasetDistribution

Output only. Dataset distributions for the user output tokens.

user_message_per_example_distribution SupervisedTuningDatasetDistribution

Output only. Dataset distributions for the messages per example.

user_dataset_examples[] Content

Output only. Sample user messages in the training dataset uri.

total_truncated_example_count int64

The number of examples in the dataset that have been truncated by any amount.

truncated_example_indices[] int64

A partial sample of the indices (starting from 1) of the truncated examples.

SupervisedTuningDatasetDistribution

Dataset distribution for Supervised Tuning.

Fields
sum int64

Output only. Sum of a given population of values.

billable_sum int64

Output only. Sum of a given population of values that are billable.

min double

Output only. The minimum of the population values.

max double

Output only. The maximum of the population values.

mean double

Output only. The arithmetic mean of the values in the population.

median double

Output only. The median of the values in the population.

p5 double

Output only. The 5th percentile of the values in the population.

p95 double

Output only. The 95th percentile of the values in the population.

buckets[] DatasetBucket

Output only. Defines the histogram bucket.

DatasetBucket

Dataset bucket used to create a histogram for the distribution given a population of values.

Fields
count double

Output only. Number of values in the bucket.

left double

Output only. Left bound of the bucket.

right double

Output only. Right bound of the bucket.

SupervisedTuningSpec

Tuning Spec for Supervised Tuning for first party models.

Fields
training_dataset_uri string

Required. Cloud Storage path to file containing training dataset for tuning. The dataset must be formatted as a JSONL file.

validation_dataset_uri string

Optional. Cloud Storage path to file containing validation dataset for tuning. The dataset must be formatted as a JSONL file.

hyper_parameters SupervisedHyperParameters

Optional. Hyperparameters for SFT.

Tensor

A tensor value type.

Fields
dtype DataType

The data type of tensor.

shape[] int64

Shape of the tensor.

bool_val[] bool

Type specific representations that make it easy to create tensor protos in all languages. Only the representation corresponding to "dtype" can be set. The values hold the flattened representation of the tensor in row major order.

[BOOL][google.aiplatform.master.Tensor.DataType.BOOL]

string_val[] string

[STRING][google.aiplatform.master.Tensor.DataType.STRING]

bytes_val[] bytes

[STRING][google.aiplatform.master.Tensor.DataType.STRING]

float_val[] float

[FLOAT][google.aiplatform.master.Tensor.DataType.FLOAT]

double_val[] double

[DOUBLE][google.aiplatform.master.Tensor.DataType.DOUBLE]

int_val[] int32

[INT_8][google.aiplatform.master.Tensor.DataType.INT8] [INT_16][google.aiplatform.master.Tensor.DataType.INT16] [INT_32][google.aiplatform.master.Tensor.DataType.INT32]

int64_val[] int64

[INT64][google.aiplatform.master.Tensor.DataType.INT64]

uint_val[] uint32

[UINT8][google.aiplatform.master.Tensor.DataType.UINT8] [UINT16][google.aiplatform.master.Tensor.DataType.UINT16] [UINT32][google.aiplatform.master.Tensor.DataType.UINT32]

uint64_val[] uint64

[UINT64][google.aiplatform.master.Tensor.DataType.UINT64]

list_val[] Tensor

A list of tensor values.

struct_val map<string, Tensor>

A map of string to tensor.

tensor_val bytes

Serialized raw tensor content.

DataType

Data type of the tensor.

Enums
DATA_TYPE_UNSPECIFIED Not a legal value for DataType. Used to indicate a DataType field has not been set.
BOOL Data types that all computation devices are expected to be capable to support.
STRING
FLOAT
DOUBLE
INT8
INT16
INT32
INT64
UINT8
UINT16
UINT32
UINT64

Tool

Tool details that the model may use to generate response.

A Tool is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. A Tool object should contain exactly one type of Tool (e.g FunctionDeclaration, Retrieval or GoogleSearchRetrieval).

Fields
function_declarations[] FunctionDeclaration

Optional. Function tool type. One or more function declarations to be passed to the model along with the current user query. Model may decide to call a subset of these functions by populating [FunctionCall][content.part.function_call] in the response. User should provide a [FunctionResponse][content.part.function_response] for each function call in the next turn. Based on the function responses, Model will generate the final response back to the user. Maximum 128 function declarations can be provided.

retrieval Retrieval

Optional. Retrieval tool type. System will always execute the provided retrieval tool(s) to get external knowledge to answer the prompt. Retrieval results are presented to the model for generation.

google_search_retrieval GoogleSearchRetrieval

Optional. GoogleSearchRetrieval tool type. Specialized retrieval tool that is powered by Google search.

ToolCallValidInput

Input for tool call valid metric.

Fields
metric_spec ToolCallValidSpec

Required. Spec for tool call valid metric.

instances[] ToolCallValidInstance

Required. Repeated tool call valid instances.

ToolCallValidInstance

Spec for tool call valid instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Required. Ground truth used to compare against the prediction.

ToolCallValidMetricValue

Tool call valid metric value for an instance.

Fields
score float

Output only. Tool call valid score.

ToolCallValidResults

Results for tool call valid metric.

Fields
tool_call_valid_metric_values[] ToolCallValidMetricValue

Output only. Tool call valid metric values.

ToolCallValidSpec

This type has no fields.

Spec for tool call valid metric.

ToolConfig

Tool config. This config is shared for all tools provided in the request.

Fields
function_calling_config FunctionCallingConfig

Optional. Function calling config.

ToolNameMatchInput

Input for tool name match metric.

Fields
metric_spec ToolNameMatchSpec

Required. Spec for tool name match metric.

instances[] ToolNameMatchInstance

Required. Repeated tool name match instances.

ToolNameMatchInstance

Spec for tool name match instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Required. Ground truth used to compare against the prediction.

ToolNameMatchMetricValue

Tool name match metric value for an instance.

Fields
score float

Output only. Tool name match score.

ToolNameMatchResults

Results for tool name match metric.

Fields
tool_name_match_metric_values[] ToolNameMatchMetricValue

Output only. Tool name match metric values.

ToolNameMatchSpec

This type has no fields.

Spec for tool name match metric.

ToolParameterKVMatchInput

Input for tool parameter key value match metric.

Fields
metric_spec ToolParameterKVMatchSpec

Required. Spec for tool parameter key value match metric.

instances[] ToolParameterKVMatchInstance

Required. Repeated tool parameter key value match instances.

ToolParameterKVMatchInstance

Spec for tool parameter key value match instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Required. Ground truth used to compare against the prediction.

ToolParameterKVMatchMetricValue

Tool parameter key value match metric value for an instance.

Fields
score float

Output only. Tool parameter key value match score.

ToolParameterKVMatchResults

Results for tool parameter key value match metric.

Fields
tool_parameter_kv_match_metric_values[] ToolParameterKVMatchMetricValue

Output only. Tool parameter key value match metric values.

ToolParameterKVMatchSpec

Spec for tool parameter key value match metric.

Fields
use_strict_string_match bool

Optional. Whether to use STRCIT string match on parameter values.

ToolParameterKeyMatchInput

Input for tool parameter key match metric.

Fields
metric_spec ToolParameterKeyMatchSpec

Required. Spec for tool parameter key match metric.

instances[] ToolParameterKeyMatchInstance

Required. Repeated tool parameter key match instances.

ToolParameterKeyMatchInstance

Spec for tool parameter key match instance.

Fields
prediction string

Required. Output of the evaluated model.

reference string

Required. Ground truth used to compare against the prediction.

ToolParameterKeyMatchMetricValue

Tool parameter key match metric value for an instance.

Fields
score float

Output only. Tool parameter key match score.

ToolParameterKeyMatchResults

Results for tool parameter key match metric.

Fields
tool_parameter_key_match_metric_values[] ToolParameterKeyMatchMetricValue

Output only. Tool parameter key match metric values.

ToolParameterKeyMatchSpec

This type has no fields.

Spec for tool parameter key match metric.

TunedModel

The Model Registry Model and Online Prediction Endpoint assiociated with this TuningJob.

Fields
model string

Output only. The resource name of the TunedModel. Format: projects/{project}/locations/{location}/models/{model}.

endpoint string

Output only. A resource name of an Endpoint. Format: projects/{project}/locations/{location}/endpoints/{endpoint}.

TunedModelRef

TunedModel Reference for legacy model migration.

Fields
Union field tuned_model_ref. The Tuned Model Reference for the model. tuned_model_ref can be only one of the following:
tuned_model string

Support migration from model registry.

tuning_job string

Support migration from tuning job list page, from gemini-1.0-pro-002 to 1.5 and above.

pipeline_job string

Support migration from tuning job list page, from bison model to gemini model.

TuningDataStats

The tuning data statistic values for TuningJob.

Fields

Union field tuning_data_stats.

tuning_data_stats can be only one of the following:

supervised_tuning_data_stats SupervisedTuningDataStats

The SFT Tuning data stats.

TuningJob

Represents a TuningJob that runs with Google owned models.

Fields
name string

Output only. Identifier. Resource name of a TuningJob. Format: projects/{project}/locations/{location}/tuningJobs/{tuning_job}

tuned_model_display_name string

Optional. The display name of the TunedModel. The name can be up to 128 characters long and can consist of any UTF-8 characters.

description string

Optional. The description of the TuningJob.

state JobState

Output only. The detailed state of the job.

create_time Timestamp

Output only. Time when the TuningJob was created.

start_time Timestamp

Output only. Time when the TuningJob for the first time entered the JOB_STATE_RUNNING state.

end_time Timestamp

Output only. Time when the TuningJob entered any of the following JobStates: JOB_STATE_SUCCEEDED, JOB_STATE_FAILED, JOB_STATE_CANCELLED, JOB_STATE_EXPIRED.

update_time Timestamp

Output only. Time when the TuningJob was most recently updated.

error Status

Output only. Only populated when job's state is JOB_STATE_FAILED or JOB_STATE_CANCELLED.

labels map<string, string>

Optional. The labels with user-defined metadata to organize TuningJob and generated resources such as Model and Endpoint.

Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed.

See https://goo.gl/xmQnxf for more information and examples of labels.

experiment string

Output only. The Experiment associated with this TuningJob.

tuned_model TunedModel

Output only. The tuned model resources assiociated with this TuningJob.

tuning_data_stats TuningDataStats

Output only. The tuning data statistics associated with this TuningJob.

encryption_spec EncryptionSpec

Customer-managed encryption key options for a TuningJob. If this is set, then all resources created by the TuningJob will be encrypted with the provided encryption key.

Union field source_model.

source_model can be only one of the following:

base_model string

The base model that is being tuned, e.g., "gemini-1.0-pro-002".

Union field tuning_spec.

tuning_spec can be only one of the following:

supervised_tuning_spec SupervisedTuningSpec

Tuning Spec for Supervised Fine Tuning.

Type

Type contains the list of OpenAPI data types as defined by https://swagger.io/docs/specification/data-models/data-types/

Enums
TYPE_UNSPECIFIED Not specified, should not be used.
STRING OpenAPI string type
NUMBER OpenAPI number type
INTEGER OpenAPI integer type
BOOLEAN OpenAPI boolean type
ARRAY OpenAPI array type
OBJECT OpenAPI object type

VertexAISearch

Retrieve from Vertex AI Search datastore for grounding. See https://cloud.google.com/products/agent-builder

Fields
datastore string

Required. Fully-qualified Vertex AI Search data store resource ID. Format: projects/{project}/locations/{location}/collections/{collection}/dataStores/{dataStore}

VertexRagStore

Retrieve from Vertex RAG Store for grounding.

Fields
rag_corpora[]
(deprecated) string

Optional. Deprecated. Please use rag_resources instead.

rag_resources[] RagResource

Optional. The representation of the rag source. It can be used to specify corpus only or ragfiles. Currently only support one corpus or multiple files from one corpus. In the future we may open up multiple corpora support.

similarity_top_k int32

Optional. Number of top k results to return from the selected corpora.

vector_distance_threshold double

Optional. Only return results with vector distance smaller than the threshold.

RagResource

The definition of the Rag resource.

Fields
rag_corpus string

Optional. RagCorpora resource name. Format: projects/{project}/locations/{location}/ragCorpora/{rag_corpus}

rag_file_ids[] string

Optional. rag_file_id. The files should be in the same rag_corpus set in rag_corpus field.

VideoMetadata

Metadata describes the input video content.

Fields
start_offset Duration

Optional. The start offset of the video.

end_offset Duration

Optional. The end offset of the video.