Class RoutingConfig (1.68.0) 

RoutingConfig(
    *,
    routing_config: typing.Union[
        vertexai.generative_models._generative_models.GenerationConfig.RoutingConfig.AutoRoutingMode,
        vertexai.generative_models._generative_models.GenerationConfig.RoutingConfig.ManualRoutingMode,
    ]
)

The configuration for model router requests.

The routing config is either one of the two nested classes:

  • AutoRoutingMode: Automated routing.
  • ManualRoutingMode: Manual routing.

Usage:

  • AutoRoutingMode:

    routing_config=generative_models.RoutingConfig(
    routing_config=generative_models.RoutingConfig.AutoRoutingMode(
        model_routing_preference=generative_models.RoutingConfig.AutoRoutingMode.ModelRoutingPreference.BALANCED,
    ),
)

  • ManualRoutingMode:

    routing_config=generative_models.RoutingConfig(
    routing_config=generative_models.RoutingConfig.ManutalRoutingMode(
        model_name="gemini-1.5-pro-001",
    ),
)

Classes

AutoRoutingMode

AutoRoutingMode(
    *,
    model_routing_preference: google.cloud.aiplatform_v1beta1.types.content.GenerationConfig.RoutingConfig.AutoRoutingMode.ModelRoutingPreference
)

When automated routing is specified, the routing will be determined by the routing model predicted quality and customer provided model routing preference.

ManualRoutingMode

ManualRoutingMode(*, model_name: str)

When manual routing is set, the specified model will be used directly.