Class GenerationConfig (1.74.0)

GenerationConfig(
    *,
    temperature: typing.Optional[float] = None,
    top_p: typing.Optional[float] = None,
    top_k: typing.Optional[int] = None,
    candidate_count: typing.Optional[int] = None,
    max_output_tokens: typing.Optional[int] = None,
    stop_sequences: typing.Optional[typing.List[str]] = None,
    presence_penalty: typing.Optional[float] = None,
    frequency_penalty: typing.Optional[float] = None,
    response_mime_type: typing.Optional[str] = None,
    response_schema: typing.Optional[typing.Dict[str, typing.Any]] = None,
    seed: typing.Optional[int] = None,
    audio_timestamp: typing.Optional[bool] = None,
    routing_config: typing.Optional[RoutingConfig] = None,
    logprobs: typing.Optional[int] = None,
    response_logprobs: typing.Optional[bool] = None
)

Parameters for the generation.

Classes

RoutingConfig

RoutingConfig(
    *,
    routing_config: typing.Union[
        vertexai.generative_models._generative_models.GenerationConfig.RoutingConfig.AutoRoutingMode,
        vertexai.generative_models._generative_models.GenerationConfig.RoutingConfig.ManualRoutingMode,
    ]
)

The configuration for model router requests.

The routing config is either one of the two nested classes:

  • AutoRoutingMode: Automated routing.
  • ManualRoutingMode: Manual routing.

Usage:

  • AutoRoutingMode:

    routing_config=generative_models.RoutingConfig(
        routing_config=generative_models.RoutingConfig.AutoRoutingMode(
            model_routing_preference=generative_models.RoutingConfig.AutoRoutingMode.ModelRoutingPreference.BALANCED,
        ),
    )
    
  • ManualRoutingMode:

    routing_config=generative_models.RoutingConfig(
        routing_config=generative_models.RoutingConfig.ManutalRoutingMode(
            model_name="gemini-1.5-pro-001",
        ),
    )
    

Methods

GenerationConfig

GenerationConfig(
    *,
    temperature: typing.Optional[float] = None,
    top_p: typing.Optional[float] = None,
    top_k: typing.Optional[int] = None,
    candidate_count: typing.Optional[int] = None,
    max_output_tokens: typing.Optional[int] = None,
    stop_sequences: typing.Optional[typing.List[str]] = None,
    presence_penalty: typing.Optional[float] = None,
    frequency_penalty: typing.Optional[float] = None,
    response_mime_type: typing.Optional[str] = None,
    response_schema: typing.Optional[typing.Dict[str, typing.Any]] = None,
    seed: typing.Optional[int] = None,
    audio_timestamp: typing.Optional[bool] = None,
    routing_config: typing.Optional[RoutingConfig] = None,
    logprobs: typing.Optional[int] = None,
    response_logprobs: typing.Optional[bool] = None
)

Constructs a GenerationConfig object.