GenerationConfig

Generation config.

Fields
stopSequences[] string

Optional. Stop sequences.

responseMimeType string

Optional. Output response mimetype of the generated candidate text. Supported mimetype: - text/plain: (default) Text output. - application/json: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.

temperature number

Optional. Controls the randomness of predictions.

topP number

Optional. If specified, nucleus sampling will be used.

topK number

Optional. If specified, top-k sampling will be used.

candidateCount integer

Optional. Number of candidates to generate.

maxOutputTokens integer

Optional. The maximum number of output tokens to generate per message.

responseLogprobs boolean

Optional. If true, export the logprobs results in response.

logprobs integer

Optional. Logit probabilities.

presencePenalty number

Optional. Positive penalties.

frequencyPenalty number

Optional. Frequency penalties.

seed integer

Optional. Seed.

responseSchema object (Schema)

Optional. The Schema object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an OpenAPI 3.0 schema object. If set, a compatible responseMimeType must also be set. Compatible mimetypes: application/json: Schema for JSON response.

routingConfig object (RoutingConfig)

Optional. Routing configuration.

JSON representation 
{
  "stopSequences": [
    string
  ],
  "responseMimeType": string,
  "temperature": number,
  "topP": number,
  "topK": number,
  "candidateCount": integer,
  "maxOutputTokens": integer,
  "responseLogprobs": boolean,
  "logprobs": integer,
  "presencePenalty": number,
  "frequencyPenalty": number,
  "seed": integer,
  "responseSchema": {
    object (Schema)
  },
  "routingConfig": {
    object (RoutingConfig)
  }
}

RoutingConfig

The configuration for routing the request to a specific model.

Fields
routing_config Union type
Routing mode. routing_config can be only one of the following:
autoMode object (AutoRoutingMode)

Automated routing.

manualMode object (ManualRoutingMode)

Manual routing.

JSON representation 
{

  // routing_config
  "autoMode": {
    object (AutoRoutingMode)
  },
  "manualMode": {
    object (ManualRoutingMode)
  }
  // Union type
}

AutoRoutingMode

When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference.

Fields
modelRoutingPreference enum (ModelRoutingPreference)

The model routing preference.

JSON representation 
{
  "modelRoutingPreference": enum (ModelRoutingPreference)
}

ModelRoutingPreference

The model routing preference.

Enums
UNKNOWN Unspecified model routing preference.
PRIORITIZE_QUALITY Prefer higher quality over low cost.
BALANCED Balanced model routing preference.
PRIORITIZE_COST Prefer lower cost over higher quality.

ManualRoutingMode

When manual routing is set, the specified model will be used directly.

Fields
modelName string

The model name to use. Only the public LLM models are accepted. e.g. 'gemini-1.5-pro-001'.

JSON representation 
{
  "modelName": string
}