GenerationConfig

Generation config.

Fields
stopSequences[] string

Optional. Stop sequences.

responseMimeType string

Optional. Output response mimetype of the generated candidate text. Supported mimetype: - text/plain: (default) Text output. - application/json: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.

thinkingConfig object (ThinkingConfig)

Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.

temperature number

Optional. Controls the randomness of predictions.

topP number

Optional. If specified, nucleus sampling will be used.

topK number

Optional. If specified, top-k sampling will be used.

candidateCount integer

Optional. Number of candidates to generate.

maxOutputTokens integer

Optional. The maximum number of output tokens to generate per message.

responseLogprobs boolean

Optional. If true, export the logprobs results in response.

logprobs integer

Optional. Logit probabilities.

presencePenalty number

Optional. Positive penalties.

frequencyPenalty number

Optional. Frequency penalties.

seed integer

Optional. Seed.

responseSchema object (Schema)

Optional. The Schema object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an OpenAPI 3.0 schema object. If set, a compatible responseMimeType must also be set. Compatible mimetypes: application/json: Schema for JSON response.

responseJsonSchema value (Value format)

Optional. Output schema of the generated response. This is an alternative to responseSchema that accepts JSON Schema.

If set, responseSchema must be omitted, but responseMimeType is required.

While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported:

  • $id
  • $defs
  • $ref
  • $anchor
  • type
  • format
  • title
  • description
  • enum (for strings and numbers)
  • items
  • prefixItems
  • minItems
  • maxItems
  • minimum
  • maximum
  • anyOf
  • oneOf (interpreted the same as anyOf)
  • properties
  • additionalProperties
  • required

The non-standard propertyOrdering property may also be set.

Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If $ref is set on a sub-schema, no other properties, except for than those starting as a $, may be set.

routingConfig object (RoutingConfig)

Optional. Routing configuration.

enableAffectiveDialog boolean

Optional. If enabled, the model will detect emotions and adapt its responses accordingly.

JSON representation 
{
  "stopSequences": [
    string
  ],
  "responseMimeType": string,
  "thinkingConfig": {
    object (ThinkingConfig)
  },
  "temperature": number,
  "topP": number,
  "topK": number,
  "candidateCount": integer,
  "maxOutputTokens": integer,
  "responseLogprobs": boolean,
  "logprobs": integer,
  "presencePenalty": number,
  "frequencyPenalty": number,
  "seed": integer,
  "responseSchema": {
    object (Schema)
  },
  "responseJsonSchema": value,
  "routingConfig": {
    object (RoutingConfig)
  },
  "enableAffectiveDialog": boolean
}

RoutingConfig

The configuration for routing the request to a specific model.

Fields
routing_config Union type
Routing mode. routing_config can be only one of the following:
autoMode object (AutoRoutingMode)

Automated routing.

manualMode object (ManualRoutingMode)

Manual routing.

JSON representation 
{

  // routing_config
  "autoMode": {
    object (AutoRoutingMode)
  },
  "manualMode": {
    object (ManualRoutingMode)
  }
  // Union type
}

AutoRoutingMode

When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference.

Fields
modelRoutingPreference enum (ModelRoutingPreference)

The model routing preference.

JSON representation 
{
  "modelRoutingPreference": enum (ModelRoutingPreference)
}

ModelRoutingPreference

The model routing preference.

Enums
UNKNOWN Unspecified model routing preference.
PRIORITIZE_QUALITY Prefer higher quality over low cost.
BALANCED Balanced model routing preference.
PRIORITIZE_COST Prefer lower cost over higher quality.

ManualRoutingMode

When manual routing is set, the specified model will be used directly.

Fields
modelName string

The model name to use. Only the public LLM models are accepted. See Supported models.

JSON representation 
{
  "modelName": string
}

ThinkingConfig

Config for thinking features.

Fields
includeThoughts boolean

Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.

thinkingBudget integer

Optional. Indicates the thinking budget in tokens. This is only applied when enableThinking is true.

JSON representation 
{
  "includeThoughts": boolean,
  "thinkingBudget": integer
}