REST Resource: projects.locations.evaluations

string

Identifier. The full resource name of the Evaluation, in the format of projects/{project}/locations/{location}/evaluations/{evaluation}.

This field must be a UTF-8 encoded string with a length limit of 1024 characters.

object (EvaluationSpec)

Required. The specification of the evaluation.

object (QualityMetrics)

Output only. The metrics produced by the evaluation, averaged across all SampleQuerys in the SampleQuerySet.

Only populated when the evaluation's state is SUCCEEDED.

enum (State)

Output only. The state of the evaluation.

object (Status)

Output only. The error that occurred during evaluation. Only populated when the evaluation's state is FAILED.

string (Timestamp format)

Output only. timestamp the Evaluation was created at.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

string (Timestamp format)

Output only. timestamp the Evaluation was completed at.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

object (Status)

Output only. A sample of errors encountered while processing the request.

object (QuerySetSpec)

Required. The specification of the query set.

object (SearchRequest)

Required. The search request that is used to perform the evaluation.

Only the following fields within SearchRequest are supported; if any other fields are provided, an UNSUPPORTED error will be returned:

• SearchRequest.serving_config
• SearchRequest.branch
• SearchRequest.canonical_filter
• SearchRequest.query_expansion_spec
• SearchRequest.spell_correction_spec
• SearchRequest.content_search_spec
• SearchRequest.user_pseudo_id

string

Base64 encoded image bytes. Supported image formats: JPEG, PNG, and BMP.

string

Required. Full resource name of DataStore, such as projects/{project}/locations/{location}/collections/{collectionId}/dataStores/{dataStoreId}.

object (FacetKey)

Required. The facet key specification.

integer

Maximum facet values that are returned for this facet. If unspecified, defaults to 20. The maximum allowed value is 300. Values above 300 are coerced to 300. For aggregation in healthcare search, when the [FacetKey.key] is "healthcare_aggregation_key", the limit will be overridden to 10,000 internally, regardless of the value set here.

If this field is negative, an INVALID_ARGUMENT is returned.

string

List of keys to exclude when faceting.

By default, FacetKey.key is not excluded from the filter unless it is listed in this field.

Listing a facet key in this field allows its values to appear as facet results, even when they are filtered out of search results. Using this field does not affect what search results are returned.

For example, suppose there are 100 documents with the color facet "Red" and 200 documents with the color facet "Blue". A query containing the filter "color:ANY("Red")" and having "color" as FacetKey.key would by default return only "Red" documents in the search results, and also return "Red" with count 100 as the only color facet. Although there are also blue documents available, "Blue" would not be shown as an available facet value.

If "color" is listed in "excludedFilterKeys", then the query returns the facet values "Red" with count 100 and "Blue" with count 200, because the "color" key is now excluded from the filter. Because this field doesn't affect search results, the search results are still correctly filtered to return only "Red" documents.

A maximum of 100 values are allowed. Otherwise, an INVALID_ARGUMENT error is returned.

boolean

Enables dynamic position for this facet. If set to true, the position of this facet among all facets in the response is determined automatically. If dynamic facets are enabled, it is ordered together. If set to false, the position of this facet in the response is the same as in the request, and it is ranked before the facets with dynamic position enable and all dynamic facets.

For example, you may always want to have rating facet returned in the response, but it's not necessarily to always display the rating facet at the top. In that case, you can set enableDynamicPosition to true so that the position of rating facet in response is determined automatically.

Another example, assuming you have the following facets in the request:

• "rating", enableDynamicPosition = true

• "price", enableDynamicPosition = false

• "brands", enableDynamicPosition = false

And also you have a dynamic facets enabled, which generates a facet gender. Then the final order of the facets in the response can be ("price", "brands", "rating", "gender") or ("price", "brands", "gender", "rating") depends on how API orders "gender" and "rating" facets. However, notice that "price" and "brands" are always ranked at first and second position because their enableDynamicPosition is false.

string

Required. Supported textual and numerical facet keys in Document object, over which the facet values are computed. Facet key is case-sensitive.

object (Interval)

Set only if values should be bucketed into intervals. Must be set for facets with numerical values. Must not be set for facet with text values. Maximum number of intervals is 30.

string

Only get facet for the given restricted values. Only supported on textual fields. For example, suppose "category" has three values "Action > 2022", "Action > 2021" and "Sci-Fi > 2022". If set "restrictedValues" to "Action > 2022", the "category" facet only contains "Action > 2022". Only supported on textual fields. Maximum is 10.

string

Only get facet values that start with the given string prefix. For example, suppose "category" has three values "Action > 2022", "Action > 2021" and "Sci-Fi > 2022". If set "prefixes" to "Action", the "category" facet only contains "Action > 2022" and "Action > 2021". Only supported on textual fields. Maximum is 10.

string

Only get facet values that contain the given strings. For example, suppose "category" has three values "Action > 2022", "Action > 2021" and "Sci-Fi > 2022". If set "contains" to "2022", the "category" facet only contains "Action > 2022" and "Sci-Fi > 2022". Only supported on textual fields. Maximum is 10.

boolean

True to make facet keys case insensitive when getting faceting values with prefixes or contains; false otherwise.

string

The order in which documents are returned.

Allowed values are:

• "count desc", which means order by SearchResponse.Facet.values.count descending.

• "value desc", which means order by SearchResponse.Facet.values.value descending. Only applies to textual facets.

If not set, textual values are sorted in natural order; numerical intervals are sorted in the order given by FacetSpec.FacetKey.intervals.

number

Inclusive lower bound.

number

Exclusive lower bound.

number

Inclusive upper bound.

number

Exclusive upper bound.

object (ConditionBoostSpec)

Condition boost specifications. If a document matches multiple conditions in the specifictions, boost scores from these specifications are all applied and combined in a non-linear way. Maximum number of specifications is 20.

string

An expression which specifies a boost condition. The syntax and supported fields are the same as a filter expression. See SearchRequest.filter for detail syntax and limitations.

Examples:

• To boost documents with document ID "doc_1" or "doc_2", and color "Red" or "Blue": (documentId: ANY("doc_1", "doc_2")) AND (color: ANY("Red", "Blue"))

number

Strength of the condition boost, which should be in [-1, 1]. Negative boost means demotion. Default is 0.0.

Setting to 1.0 gives the document a big promotion. However, it does not necessarily mean that the boosted document will be the top result at all times, nor that other documents will be excluded. Results could still be shown even when none of them matches the condition. And results that are significantly more relevant to the search query can still trump your heavily favored but irrelevant documents.

Setting to -1.0 gives the document a big demotion. However, results that are deeply relevant might still be shown. The document will have an upstream battle to get a fairly high ranking, but it is not blocked out completely.

Setting to 0.0 means no boost applied. The boosting condition is ignored. Only one of the (condition, boost) combination or the boostControlSpec below are set. If both are set then the global boost is ignored and the more fine-grained boostControlSpec is applied.

object (BoostControlSpec)

Complex specification for custom ranking based on customer defined attribute value.

string

The name of the field whose value will be used to determine the boost amount.

enum (AttributeType)

The attribute type to be used to determine the boost amount. The attribute value can be derived from the field value of the specified fieldName. In the case of numerical it is straightforward i.e. attributeValue = numerical_field_value. In the case of freshness however, attributeValue = (time.now() - datetime_field_value).

enum (InterpolationType)

The interpolation type to be applied to connect the control points listed below.

object (ControlPoint)

The control points used to define the curve. The monotonic function (defined through the interpolationType above) passes through the control points listed here.

string

Can be one of: 1. The numerical field value. 2. The duration spec for freshness: The value must be formatted as an XSD dayTimeDuration value (a restricted subset of an ISO 8601 duration value). The pattern for this is: [nD][T[nH][nM][nS]].

number

The value between -1 to 1 by which to boost the score if the attributeValue evaluates to the value specified above.

enum (Condition)

The condition under which query expansion should occur. Default to Condition.DISABLED.

boolean

Whether to pin unexpanded results. If this field is set to true, unexpanded products are always at the top of the search results, followed by the expanded results.

enum (Mode)

The mode under which spell correction replaces the original search query. Defaults to Mode.AUTO.

object (EmbeddingVector)

The embedding vector used for retrieval. Limit to 1.

string

Embedding field path in schema.

number

Query embedding vector.

enum (FilterExtractionCondition)

The condition under which filter extraction should occur. Default to [Condition.DISABLED][].

string

Field names used for location-based filtering, where geolocation filters are detected in natural language search queries. Only valid when the FilterExtractionCondition is set to ENABLED.

If this field is set, it overrides the field names set in ServingConfig.geo_search_query_detection_field_names.

enum (Condition)

The condition under which search as you type should occur. Default to Condition.DISABLED.

string

If set, the search result gets stored to the "turn" specified by this query ID.

Example: Let's say the session looks like this: session { name: ".../sessions/xxx" turns { query { text: "What is foo?" queryId: ".../questions/yyy" } answer: "Foo is ..." } turns { query { text: "How about bar then?" queryId: ".../questions/zzz" } } }

The user can call /search API with a request like this:

session: ".../sessions/xxx" sessionSpec { queryId: ".../questions/zzz" }

Then, the API stores the search result, associated with the last turn. The stored search result can be used by a subsequent /answer API call (with the session ID and the query ID specified). Also, it is possible to call /search and /answer in parallel with the same session ID & query ID.

integer

The number of top search results to persist. The persisted search results can be used for the subsequent /answer api call.

This field is simliar to the summaryResultCount field in SearchRequest.ContentSearchSpec.SummarySpec.summary_result_count.

At most 10 results for documents mode, or 50 for chunks mode.

string

Required. The full resource name of the SampleQuerySet used for the evaluation, in the format of projects/{project}/locations/{location}/sampleQuerySets/{sampleQuerySet}.

object (TopkMetrics)

Recall per document, at various top-k cutoff levels.

Recall is the fraction of relevant documents retrieved out of all relevant documents.

Example (top-5): * For a single SampleQuery, If 3 out of 5 relevant documents are retrieved in the top-5, recall@5 = 3/5 = 0.6

object (TopkMetrics)

Precision per document, at various top-k cutoff levels.

Precision is the fraction of retrieved documents that are relevant.

Example (top-5): * For a single SampleQuery, If 4 out of 5 retrieved documents in the top-5 are relevant, precision@5 = 4/5 = 0.8

object (TopkMetrics)

Normalized discounted cumulative gain (NDCG) per document, at various top-k cutoff levels.

NDCG measures the ranking quality, giving higher relevance to top results.

Example (top-3): Suppose SampleQuery with three retrieved documents (D1, D2, D3) and binary relevance judgements (1 for relevant, 0 for not relevant):

Retrieved: [D3 (0), D1 (1), D2 (1)] Ideal: [D1 (1), D2 (1), D3 (0)]

Calculate NDCG@3 for each SampleQuery: * DCG@3: 0/log2(1+1) + 1/log2(2+1) + 1/log2(3+1) = 1.13 * Ideal DCG@3: 1/log2(1+1) + 1/log2(2+1) + 0/log2(3+1) = 1.63 * NDCG@3: 1.13/1.63 = 0.693

object (TopkMetrics)

Recall per page, at various top-k cutoff levels.

Recall is the fraction of relevant pages retrieved out of all relevant pages.

Example (top-5): * For a single SampleQuery, if 3 out of 5 relevant pages are retrieved in the top-5, recall@5 = 3/5 = 0.6

object (TopkMetrics)

Normalized discounted cumulative gain (NDCG) per page, at various top-k cutoff levels.

NDCG measures the ranking quality, giving higher relevance to top results.

Example (top-3): Suppose SampleQuery with three retrieved pages (P1, P2, P3) and binary relevance judgements (1 for relevant, 0 for not relevant):

Retrieved: [P3 (0), P1 (1), P2 (1)] Ideal: [P1 (1), P2 (1), P3 (0)]

Calculate NDCG@3 for SampleQuery: * DCG@3: 0/log2(1+1) + 1/log2(2+1) + 1/log2(3+1) = 1.13 * Ideal DCG@3: 1/log2(1+1) + 1/log2(2+1) + 0/log2(3+1) = 1.63 * NDCG@3: 1.13/1.63 = 0.693

number

The top-1 value.

number

The top-3 value.

number

The top-5 value.

number

The top-10 value.