Starting April 29, 2025, Gemini 1.5 Pro and Gemini 1.5 Flash models are not available in projects that have no prior usage of these models, including new projects. For details, see Model versions and lifecycle.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-05 UTC."],[],[],null,["The metric used for running evaluations.\nFields `aggregationMetrics[]` `enum (`[AggregationMetric](/vertex-ai/generative-ai/docs/reference/rest/v1/Metric#AggregationMetric)`)` \nOptional. The aggregation metrics to use. \n`metric_spec` `Union type` \nThe spec for the metric. It would be either a pre-defined metric, or a inline metric spec. `metric_spec` can be only one of the following:\n`predefinedMetricSpec` `object (`[PredefinedMetricSpec](/vertex-ai/generative-ai/docs/reference/rest/v1/PredefinedMetricSpec)`)` \nThe spec for a pre-defined metric.\n`llmBasedMetricSpec` `object (`[LLMBasedMetricSpec](/vertex-ai/generative-ai/docs/reference/rest/v1/Metric#LLMBasedMetricSpec)`)` \nSpec for an LLM based metric.\n`pointwiseMetricSpec` `object (`[PointwiseMetricSpec](/vertex-ai/generative-ai/docs/reference/rest/v1/PointwiseMetricSpec)`)` \nSpec for pointwise metric.\n`pairwiseMetricSpec` `object (`[PairwiseMetricSpec](/vertex-ai/generative-ai/docs/reference/rest/v1/PairwiseMetricSpec)`)` \nSpec for pairwise metric.\n`exactMatchSpec` `object (`[ExactMatchSpec](/vertex-ai/generative-ai/docs/reference/rest/v1/ExactMatchSpec)`)` \nSpec for exact match metric.\n`bleuSpec` `object (`[BleuSpec](/vertex-ai/generative-ai/docs/reference/rest/v1/BleuSpec)`)` \nSpec for bleu metric.\n`rougeSpec` `object (`[RougeSpec](/vertex-ai/generative-ai/docs/reference/rest/v1/RougeSpec)`)` \nSpec for rouge metric. \n\n| JSON representation |\n|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| ``` { \"aggregationMetrics\": [ enum (/vertex-ai/generative-ai/docs/reference/rest/v1/Metric#AggregationMetric) ], // metric_spec \"predefinedMetricSpec\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/PredefinedMetricSpec) }, \"llmBasedMetricSpec\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/Metric#LLMBasedMetricSpec) }, \"pointwiseMetricSpec\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/PointwiseMetricSpec) }, \"pairwiseMetricSpec\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/PairwiseMetricSpec) }, \"exactMatchSpec\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/ExactMatchSpec) }, \"bleuSpec\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/BleuSpec) }, \"rougeSpec\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/RougeSpec) } // Union type } ``` |\n\nLLMBasedMetricSpec \nSpecification for an LLM based metric.\nFields \n`rubrics_source` `Union type` \nSource of the rubrics to be used for evaluation. `rubrics_source` can be only one of the following:\n`rubricGroupKey` `string` \nUse a pre-defined group of rubrics associated with the input. Refers to a key in the rubricGroups map of EvaluationInstance.\n`rubricGenerationSpec` `object (`[RubricGenerationSpec](/vertex-ai/generative-ai/docs/reference/rest/v1/RubricGenerationSpec)`)` \nDynamically generate rubrics using this specification.\n`predefinedRubricGenerationSpec` `object (`[PredefinedMetricSpec](/vertex-ai/generative-ai/docs/reference/rest/v1/PredefinedMetricSpec)`)` \nDynamically generate rubrics using a predefined spec.\n`metricPromptTemplate` `string` \nRequired. Template for the prompt sent to the judge model.\n`systemInstruction` `string` \nOptional. System instructions for the judge model.\n`judgeAutoraterConfig` `object (`[AutoraterConfig](/vertex-ai/generative-ai/docs/reference/rest/v1/AutoraterConfig)`)` \nOptional. Optional configuration for the judge LLM (Autorater).\n`additionalConfig` `object (`[Struct](https://protobuf.dev/reference/protobuf/google.protobuf/#struct)` format)` \nOptional. Optional additional configuration for the metric. \n\n| JSON representation |\n|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| ``` { // rubrics_source \"rubricGroupKey\": string, \"rubricGenerationSpec\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/RubricGenerationSpec) }, \"predefinedRubricGenerationSpec\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/PredefinedMetricSpec) } // Union type \"metricPromptTemplate\": string, \"systemInstruction\": string, \"judgeAutoraterConfig\": { object (/vertex-ai/generative-ai/docs/reference/rest/v1/AutoraterConfig) }, \"additionalConfig\": { object } } ``` |\n\nAggregationMetric The aggregation metrics supported by EvaluationService.EvaluateDataset.\n\n| Enums ||\n|----------------------------------|---------------------------------------------------------------------------|\n| `AGGREGATION_METRIC_UNSPECIFIED` | Unspecified aggregation metric. |\n| `AVERAGE` | Average aggregation metric. Not supported for Pairwise metric. |\n| `MODE` | Mode aggregation metric. |\n| `STANDARD_DEVIATION` | Standard deviation aggregation metric. Not supported for pairwise metric. |\n| `VARIANCE` | Variance aggregation metric. Not supported for pairwise metric. |\n| `MINIMUM` | Minimum aggregation metric. Not supported for pairwise metric. |\n| `MAXIMUM` | Maximum aggregation metric. Not supported for pairwise metric. |\n| `MEDIAN` | Median aggregation metric. Not supported for pairwise metric. |\n| `PERCENTILE_P90` | 90th percentile aggregation metric. Not supported for pairwise metric. |\n| `PERCENTILE_P95` | 95th percentile aggregation metric. Not supported for pairwise metric. |\n| `PERCENTILE_P99` | 99th percentile aggregation metric. Not supported for pairwise metric. |"]]
