Resource: ModelEvaluationSlice
A collection of metrics calculated by comparing Model's predictions on a slice of the test data against ground truth annotations.
name
string
Output only. The resource name of the ModelEvaluationSlice.
Output only. The slice of the test data that is used to evaluate the Model.
metricsSchemaUri
string
Output only. Points to a YAML file stored on Google Cloud Storage describing the metrics
of this ModelEvaluationSlice. The schema is defined as an OpenAPI 3.0.2 Schema Object.
Output only. Sliced evaluation metrics of the Model. The schema of the metrics is stored in metricsSchemaUri
Output only. timestamp when this ModelEvaluationSlice was created.
A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z"
and "2014-10-02T15:01:23.045123456Z"
.
Output only. Aggregated explanation metrics for the Model's prediction output over the data this ModelEvaluation uses. This field is populated only if the Model is evaluated with explanations, and only for tabular Models.
JSON representation |
---|
{ "name": string, "slice": { object ( |
Slice
Definition of a slice.
dimension
string
Output only. The dimension of the slice. Well-known dimensions are: * annotationSpec
: This slice is on the test data that has either ground truth or prediction with AnnotationSpec.display_name
equals to value
. * slice
: This slice is a user customized slice defined by its SliceSpec.
value
string
Output only. The value of the dimension in this slice.
Output only. Specification for how the data was sliced.
JSON representation |
---|
{
"dimension": string,
"value": string,
"sliceSpec": {
object ( |
SliceSpec
Specification for how the data should be sliced.
Mapping configuration for this SliceSpec. The key is the name of the feature. By default, the key will be prefixed by "instance" as a dictionary prefix for Vertex Batch Predictions output format.
JSON representation |
---|
{
"configs": {
string: {
object ( |
SliceConfig
Specification message containing the config for this SliceSpec. When kind
is selected as value
and/or range
, only a single slice will be computed. When allValues
is present, a separate slice will be computed for each possible label/value for the corresponding key in config
. Examples, with feature zip_code with values 12345, 23334, 88888 and feature country with values "US", "Canada", "Mexico" in the dataset:
Example 1:
{
"zip_code": { "value": { "floatValue": 12345.0 } }
}
A single slice for any data with zip_code 12345 in the dataset.
Example 2:
{
"zip_code": { "range": { "low": 12345, "high": 20000 } }
}
A single slice containing data where the zip_codes between 12345 and 20000 For this example, data with the zip_code of 12345 will be in this slice.
Example 3:
{
"zip_code": { "range": { "low": 10000, "high": 20000 } },
"country": { "value": { "stringValue": "US" } }
}
A single slice containing data where the zip_codes between 10000 and 20000 has the country "US". For this example, data with the zip_code of 12345 and country "US" will be in this slice.
Example 4:
{ "country": {"allValues": { "value": true } } }
Three slices are computed, one for each unique country in the dataset.
Example 5:
{
"country": { "allValues": { "value": true } },
"zip_code": { "value": { "floatValue": 12345.0 } }
}
Three slices are computed, one for each unique country in the dataset where the zip_code is also 12345. For this example, data with zip_code 12345 and country "US" will be in one slice, zip_code 12345 and country "Canada" in another slice, and zip_code 12345 and country "Mexico" in another slice, totaling 3 slices.
kind
Union type
kind
can be only one of the following:A unique specific value for a given feature. Example: { "value": { "stringValue": "12345" } }
A range of values for a numerical feature. Example: {"range":{"low":10000.0,"high":50000.0}}
will capture 12345 and 23334 in the slice.
allValues
boolean
If allValues is set to true, then all possible labels of the keyed feature will have another slice computed. Example: {"allValues":{"value":true}}
Value
Single value that supports strings and floats.
kind
Union type
kind
can be only one of the following:stringValue
string
String type.
floatValue
number
Float type.
JSON representation |
---|
{ // kind "stringValue": string, "floatValue": number // Union type } |
Range
A range of values for slice(s). low
is inclusive, high
is exclusive.
low
number
Inclusive low value for the range.
high
number
Exclusive high value for the range.
JSON representation |
---|
{ "low": number, "high": number } |
Methods |
|
---|---|
|
Imports a list of externally generated EvaluatedAnnotations. |
|
Gets a ModelEvaluationSlice. |
|
Lists ModelEvaluationSlices in a ModelEvaluation. |