REST Resource: projects.locations.evaluations

Resource: Evaluation

An evaluation is a single execution (or run) of an evaluation process. It encapsulates the state of the evaluation and the resulting data.

JSON representation 
{
  "name": string,
  "evaluationSpec": {
    object (EvaluationSpec)
  },
  "qualityMetrics": {
    object (QualityMetrics)
  },
  "state": enum (State),
  "error": {
    object (Status)
  },
  "createTime": string,
  "endTime": string,
  "errorSamples": [
    {
      object (Status)
    }
  ]
}
Fields
name

string

Identifier. The full resource name of the Evaluation, in the format of projects/{project}/locations/{location}/evaluations/{evaluation}.

This field must be a UTF-8 encoded string with a length limit of 1024 characters.
evaluationSpec

object (EvaluationSpec)

Required. The specification of the evaluation.
qualityMetrics

object (QualityMetrics)

Output only. The metrics produced by the evaluation, averaged across all SampleQuerys in the SampleQuerySet.

Only populated when the evaluation's state is SUCCEEDED.
state

enum (State)

Output only. The state of the evaluation.
error

object (Status)

Output only. The error that occurred during evaluation. Only populated when the evaluation's state is FAILED.
createTime

string (Timestamp format)

Output only. timestamp the Evaluation was created at.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
endTime

string (Timestamp format)

Output only. timestamp the Evaluation was completed at.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
errorSamples[]

object (Status)

Output only. A sample of errors encountered while processing the request.

EvaluationSpec

Describes the specification of the evaluation.

JSON representation 
{
  "querySetSpec": {
    object (QuerySetSpec)
  },

  // Union field search_spec can be only one of the following:
  "searchRequest": {
    object (SearchRequest)
  }
  // End of list of possible types for union field search_spec.
}
Fields
querySetSpec

object (QuerySetSpec)

Required. The specification of the query set.
Union field search_spec. The search specification. search_spec can be only one of the following:
searchRequest

object (SearchRequest)

Required. The search request that is used to perform the evaluation.

Only the following fields within SearchRequest are supported; if any other fields are provided, an UNSUPPORTED error will be returned:

SearchRequest

Request message for SearchService.Search method.

JSON representation 
{
  "servingConfig": string,
  "branch": string,
  "query": string,
  "imageQuery": {
    object (ImageQuery)
  },
  "pageSize": integer,
  "pageToken": string,
  "offset": integer,
  "dataStoreSpecs": [
    {
      object (DataStoreSpec)
    }
  ],
  "filter": string,
  "canonicalFilter": string,
  "orderBy": string,
  "userInfo": {
    object (UserInfo)
  },
  "languageCode": string,
  "regionCode": string,
  "facetSpecs": [
    {
      object (FacetSpec)
    }
  ],
  "boostSpec": {
    object (BoostSpec)
  },
  "params": {
    string: value,
    ...
  },
  "queryExpansionSpec": {
    object (QueryExpansionSpec)
  },
  "spellCorrectionSpec": {
    object (SpellCorrectionSpec)
  },
  "userPseudoId": string,
  "contentSearchSpec": {
    object (ContentSearchSpec)
  },
  "embeddingSpec": {
    object (EmbeddingSpec)
  },
  "rankingExpression": string,
  "safeSearch": boolean,
  "userLabels": {
    string: string,
    ...
  },
  "naturalLanguageQueryUnderstandingSpec": {
    object (NaturalLanguageQueryUnderstandingSpec)
  },
  "searchAsYouTypeSpec": {
    object (SearchAsYouTypeSpec)
  },
  "session": string,
  "sessionSpec": {
    object (SessionSpec)
  },
  "relevanceThreshold": enum (RelevanceThreshold)
}
Fields
servingConfig

string

Required. The resource name of the Search serving config, such as projects/*/locations/global/collections/default_collection/engines/*/servingConfigs/default_serving_config, or projects/*/locations/global/collections/default_collection/dataStores/default_data_store/servingConfigs/default_serving_config. This field is used to identify the serving configuration name, set of models used to make the search.
branch

string

The branch resource name, such as projects/*/locations/global/collections/default_collection/dataStores/default_data_store/branches/0.

Use default_branch as the branch ID or leave this field empty, to search documents under the default branch.
query

string

Raw search query.
imageQuery

object (ImageQuery)

Raw image query.
pageSize

integer

Maximum number of Documents to return. The maximum allowed value depends on the data type. Values above the maximum value are coerced to the maximum value.

  • Websites with basic indexing: Default 10, Maximum 25.
  • Websites with advanced indexing: Default 25, Maximum 50.
  • Other: Default 50, Maximum 100.

If this field is negative, an INVALID_ARGUMENT is returned.
pageToken

string

A page token received from a previous SearchService.Search call. Provide this to retrieve the subsequent page.

When paginating, all other parameters provided to SearchService.Search must match the call that provided the page token. Otherwise, an INVALID_ARGUMENT error is returned.
offset

integer

A 0-indexed integer that specifies the current offset (that is, starting result location, amongst the Documents deemed by the API as relevant) in search results. This field is only considered if pageToken is unset.

If this field is negative, an INVALID_ARGUMENT is returned.
dataStoreSpecs[]

object (DataStoreSpec)

Specs defining dataStores to filter on in a search call and configurations for those dataStores. This is only considered for engines with multiple dataStores use case. For single dataStore within an engine, they should use the specs at the top level.
filter

string

The filter syntax consists of an expression language for constructing a predicate from one or more fields of the documents being filtered. Filter expression is case-sensitive.

If this field is unrecognizable, an INVALID_ARGUMENT is returned.

Filtering in Vertex AI Search is done by mapping the LHS filter key to a key property defined in the Vertex AI Search backend -- this mapping is defined by the customer in their schema. For example a media customer might have a field 'name' in their schema. In this case the filter would look like this: filter --> name:'ANY("king kong")'

For more information about filtering including syntax and filter operators, see Filter
canonicalFilter

string

The default filter that is applied when a user performs a search without checking any filters on the search page.

The filter applied to every search request when quality improvement such as query expansion is needed. In the case a query does not have a sufficient amount of results this filter will be used to determine whether or not to enable the query expansion flow. The original filter will still be used for the query expanded search. This field is strongly recommended to achieve high search quality.

For more information about filter syntax, see SearchRequest.filter.
orderBy

string

The order in which documents are returned. Documents can be ordered by a field in an Document object. Leave it unset if ordered by relevance. orderBy expression is case-sensitive.

For more information on ordering the website search results, see Order web search results. For more information on ordering the healthcare search results, see Order healthcare search results. If this field is unrecognizable, an INVALID_ARGUMENT is returned.
userInfo

object (UserInfo)

Information about the end user. Highly recommended for analytics. UserInfo.user_agent is used to deduce deviceType for analytics.
languageCode

string

The BCP-47 language code, such as "en-US" or "sr-Latn". For more information, see Standard fields. This field helps to better interpret the query. If a value isn't specified, the query language code is automatically detected, which may not be accurate.
regionCode

string

The Unicode country/region code (CLDR) of a location, such as "US" and "419". For more information, see Standard fields. If set, then results will be boosted based on the regionCode provided.
facetSpecs[]

object (FacetSpec)

Facet specifications for faceted search. If empty, no facets are returned.

A maximum of 100 values are allowed. Otherwise, an INVALID_ARGUMENT error is returned.
boostSpec

object (BoostSpec)

Boost specification to boost certain documents. For more information on boosting, see Boosting
params

map (key: string, value: value (Value format))

Additional search parameters.

For public website search only, supported values are:

  • user_country_code: string. Default empty. If set to non-empty, results are restricted or boosted based on the location provided. For example, user_country_code: "au"

For available codes see Country Codes

  • searchType: double. Default empty. Enables non-webpage searching depending on the value. The only valid non-default value is 1, which enables image searching. For example, searchType: 1
queryExpansionSpec

object (QueryExpansionSpec)

The query expansion specification that specifies the conditions under which query expansion occurs.
spellCorrectionSpec

object (SpellCorrectionSpec)

The spell correction specification that specifies the mode under which spell correction takes effect.
userPseudoId

string

A unique identifier for tracking visitors. For example, this could be implemented with an HTTP cookie, which should be able to uniquely identify a visitor on a single device. This unique identifier should not change if the visitor logs in or out of the website.

This field should NOT have a fixed value such as unknown_visitor.

This should be the same identifier as UserEvent.user_pseudo_id and CompleteQueryRequest.user_pseudo_id

The field must be a UTF-8 encoded string with a length limit of 128 characters. Otherwise, an INVALID_ARGUMENT error is returned.
contentSearchSpec

object (ContentSearchSpec)

A specification for configuring the behavior of content search.
embeddingSpec

object (EmbeddingSpec)

Uses the provided embedding to do additional semantic document retrieval. The retrieval is based on the dot product of SearchRequest.EmbeddingSpec.EmbeddingVector.vector and the document embedding that is provided in SearchRequest.EmbeddingSpec.EmbeddingVector.field_path.

If SearchRequest.EmbeddingSpec.EmbeddingVector.field_path is not provided, it will use ServingConfig.EmbeddingConfig.field_path.
rankingExpression

string

The ranking expression controls the customized ranking on retrieval documents. This overrides ServingConfig.ranking_expression. The ranking expression is a single function or multiple functions that are joined by "+".

  • rankingExpression = function, { " + ", function };

Supported functions:

  • double * relevanceScore
  • double * dotProduct(embedding_field_path)

Function variables:

  • relevanceScore: pre-defined keywords, used for measure relevance between query and document.
  • embedding_field_path: the document embedding field used with query embedding vector.
  • dotProduct: embedding function between embedding_field_path and query embedding vector.

Example ranking expression:

If document has an embedding field doc_embedding, the ranking expression could be 0.5 * relevanceScore + 0.3 * dotProduct(doc_embedding).
userLabels

map (key: string, value: string)

The user labels applied to a resource must meet the following requirements:

  • Each resource can have multiple labels, up to a maximum of 64.
  • Each label must be a key-value pair.
  • Keys have a minimum length of 1 character and a maximum length of 63 characters and cannot be empty. Values can be empty and have a maximum length of 63 characters.
  • Keys and values can contain only lowercase letters, numeric characters, underscores, and dashes. All characters must use UTF-8 encoding, and international characters are allowed.
  • The key portion of a label must be unique. However, you can use the same key with multiple resources.
  • Keys must start with a lowercase letter or international character.

See Google Cloud Document for more details.
naturalLanguageQueryUnderstandingSpec

object (NaturalLanguageQueryUnderstandingSpec)

If naturalLanguageQueryUnderstandingSpec is not specified, no additional natural language query understanding will be done.
searchAsYouTypeSpec

object (SearchAsYouTypeSpec)

Search as you type configuration. Only supported for the IndustryVertical.MEDIA vertical.
session

string

The session resource name. Optional.

Session allows users to do multi-turn /search API calls or coordination between /search API calls and /answer API calls.

Example #1 (multi-turn /search API calls): 1. Call /search API with the auto-session mode (see below). 2. Call /search API with the session ID generated in the first call. Here, the previous search query gets considered in query standing. I.e., if the first query is "How did Alphabet do in 2022?" and the current query is "How about 2023?", the current query will be interpreted as "How did Alphabet do in 2023?".

Example #2 (coordination between /search API calls and /answer API calls): 1. Call /search API with the auto-session mode (see below). 2. Call /answer API with the session ID generated in the first call. Here, the answer generation happens in the context of the search results from the first search call.

Auto-session mode: when projects/.../sessions/- is used, a new session gets automatically created. Otherwise, users can use the create-session API to create a session manually.

Multi-turn Search feature is currently at private GA stage. Please use v1alpha or v1beta version instead before we launch this feature to public GA. Or ask for allowlisting through Google Support team.
sessionSpec

object (SessionSpec)

Session specification.

Can be used only when session is set.
relevanceThreshold

enum (RelevanceThreshold)

The relevance threshold of the search results.

Default to Google defined threshold, leveraging a balance of precision and recall to deliver both highly accurate results and comprehensive coverage of relevant information.

ImageQuery

Specifies the image query input.

JSON representation 
{

  // Union field image can be only one of the following:
  "imageBytes": string
  // End of list of possible types for union field image.
}
Fields

Union field image.

image can be only one of the following:
imageBytes

string

Base64 encoded image bytes. Supported image formats: JPEG, PNG, and BMP.

DataStoreSpec

A struct to define data stores to filter on in a search call and configurations for those data stores. Otherwise, an INVALID_ARGUMENT error is returned.

JSON representation 
{
  "dataStore": string
}
Fields
dataStore

string

Required. Full resource name of DataStore, such as projects/{project}/locations/{location}/collections/{collectionId}/dataStores/{dataStoreId}.

FacetSpec

A facet specification to perform faceted search.

JSON representation 
{
  "facetKey": {
    object (FacetKey)
  },
  "limit": integer,
  "excludedFilterKeys": [
    string
  ],
  "enableDynamicPosition": boolean
}
Fields
facetKey

object (FacetKey)

Required. The facet key specification.
limit

integer

Maximum facet values that are returned for this facet. If unspecified, defaults to 20. The maximum allowed value is 300. Values above 300 are coerced to 300. For aggregation in healthcare search, when the [FacetKey.key] is "healthcare_aggregation_key", the limit will be overridden to 10,000 internally, regardless of the value set here.

If this field is negative, an INVALID_ARGUMENT is returned.
excludedFilterKeys[]

string

List of keys to exclude when faceting.

By default, FacetKey.key is not excluded from the filter unless it is listed in this field.

Listing a facet key in this field allows its values to appear as facet results, even when they are filtered out of search results. Using this field does not affect what search results are returned.

For example, suppose there are 100 documents with the color facet "Red" and 200 documents with the color facet "Blue". A query containing the filter "color:ANY("Red")" and having "color" as FacetKey.key would by default return only "Red" documents in the search results, and also return "Red" with count 100 as the only color facet. Although there are also blue documents available, "Blue" would not be shown as an available facet value.

If "color" is listed in "excludedFilterKeys", then the query returns the facet values "Red" with count 100 and "Blue" with count 200, because the "color" key is now excluded from the filter. Because this field doesn't affect search results, the search results are still correctly filtered to return only "Red" documents.

A maximum of 100 values are allowed. Otherwise, an INVALID_ARGUMENT error is returned.
enableDynamicPosition

boolean

Enables dynamic position for this facet. If set to true, the position of this facet among all facets in the response is determined automatically. If dynamic facets are enabled, it is ordered together. If set to false, the position of this facet in the response is the same as in the request, and it is ranked before the facets with dynamic position enable and all dynamic facets.

For example, you may always want to have rating facet returned in the response, but it's not necessarily to always display the rating facet at the top. In that case, you can set enableDynamicPosition to true so that the position of rating facet in response is determined automatically.

Another example, assuming you have the following facets in the request:

  • "rating", enableDynamicPosition = true

  • "price", enableDynamicPosition = false

  • "brands", enableDynamicPosition = false

And also you have a dynamic facets enabled, which generates a facet gender. Then the final order of the facets in the response can be ("price", "brands", "rating", "gender") or ("price", "brands", "gender", "rating") depends on how API orders "gender" and "rating" facets. However, notice that "price" and "brands" are always ranked at first and second position because their enableDynamicPosition is false.

FacetKey

Specifies how a facet is computed.

JSON representation 
{
  "key": string,
  "intervals": [
    {
      object (Interval)
    }
  ],
  "restrictedValues": [
    string
  ],
  "prefixes": [
    string
  ],
  "contains": [
    string
  ],
  "caseInsensitive": boolean,
  "orderBy": string
}
Fields
key

string

Required. Supported textual and numerical facet keys in Document object, over which the facet values are computed. Facet key is case-sensitive.
intervals[]

object (Interval)

Set only if values should be bucketed into intervals. Must be set for facets with numerical values. Must not be set for facet with text values. Maximum number of intervals is 30.
restrictedValues[]

string

Only get facet for the given restricted values. Only supported on textual fields. For example, suppose "category" has three values "Action > 2022", "Action > 2021" and "Sci-Fi > 2022". If set "restrictedValues" to "Action > 2022", the "category" facet only contains "Action > 2022". Only supported on textual fields. Maximum is 10.
prefixes[]

string

Only get facet values that start with the given string prefix. For example, suppose "category" has three values "Action > 2022", "Action > 2021" and "Sci-Fi > 2022". If set "prefixes" to "Action", the "category" facet only contains "Action > 2022" and "Action > 2021". Only supported on textual fields. Maximum is 10.
contains[]

string

Only get facet values that contain the given strings. For example, suppose "category" has three values "Action > 2022", "Action > 2021" and "Sci-Fi > 2022". If set "contains" to "2022", the "category" facet only contains "Action > 2022" and "Sci-Fi > 2022". Only supported on textual fields. Maximum is 10.
caseInsensitive

boolean

True to make facet keys case insensitive when getting faceting values with prefixes or contains; false otherwise.
orderBy

string

The order in which documents are returned.

Allowed values are:

If not set, textual values are sorted in natural order; numerical intervals are sorted in the order given by FacetSpec.FacetKey.intervals.

Interval

A floating point interval.

JSON representation 
{

  // Union field min can be only one of the following:
  "minimum": number,
  "exclusiveMinimum": number
  // End of list of possible types for union field min.

  // Union field max can be only one of the following:
  "maximum": number,
  "exclusiveMaximum": number
  // End of list of possible types for union field max.
}
Fields

Union field min. The lower bound of the interval. If neither of the min fields are set, then the lower bound is negative infinity.

This field must be not larger than max. Otherwise, an INVALID_ARGUMENT error is returned. min can be only one of the following:
minimum

number

Inclusive lower bound.
exclusiveMinimum

number

Exclusive lower bound.

Union field max. The upper bound of the interval. If neither of the max fields are set, then the upper bound is positive infinity.

This field must be not smaller than min. Otherwise, an INVALID_ARGUMENT error is returned. max can be only one of the following:
maximum

number

Inclusive upper bound.
exclusiveMaximum

number

Exclusive upper bound.

BoostSpec

Boost specification to boost certain documents.

JSON representation 
{
  "conditionBoostSpecs": [
    {
      object (ConditionBoostSpec)
    }
  ]
}
Fields
conditionBoostSpecs[]

object (ConditionBoostSpec)

Condition boost specifications. If a document matches multiple conditions in the specifictions, boost scores from these specifications are all applied and combined in a non-linear way. Maximum number of specifications is 20.

ConditionBoostSpec

Boost applies to documents which match a condition.

JSON representation 
{
  "condition": string,
  "boost": number,
  "boostControlSpec": {
    object (BoostControlSpec)
  }
}
Fields
condition

string

An expression which specifies a boost condition. The syntax and supported fields are the same as a filter expression. See SearchRequest.filter for detail syntax and limitations.

Examples:

  • To boost documents with document ID "doc_1" or "doc_2", and color "Red" or "Blue": (documentId: ANY("doc_1", "doc_2")) AND (color: ANY("Red", "Blue"))
boost

number

Strength of the condition boost, which should be in [-1, 1]. Negative boost means demotion. Default is 0.0.

Setting to 1.0 gives the document a big promotion. However, it does not necessarily mean that the boosted document will be the top result at all times, nor that other documents will be excluded. Results could still be shown even when none of them matches the condition. And results that are significantly more relevant to the search query can still trump your heavily favored but irrelevant documents.

Setting to -1.0 gives the document a big demotion. However, results that are deeply relevant might still be shown. The document will have an upstream battle to get a fairly high ranking, but it is not blocked out completely.

Setting to 0.0 means no boost applied. The boosting condition is ignored. Only one of the (condition, boost) combination or the boostControlSpec below are set. If both are set then the global boost is ignored and the more fine-grained boostControlSpec is applied.
boostControlSpec

object (BoostControlSpec)

Complex specification for custom ranking based on customer defined attribute value.

BoostControlSpec

Specification for custom ranking based on customer specified attribute value. It provides more controls for customized ranking than the simple (condition, boost) combination above.

JSON representation 
{
  "fieldName": string,
  "attributeType": enum (AttributeType),
  "interpolationType": enum (InterpolationType),
  "controlPoints": [
    {
      object (ControlPoint)
    }
  ]
}
Fields
fieldName

string

The name of the field whose value will be used to determine the boost amount.
attributeType

enum (AttributeType)

The attribute type to be used to determine the boost amount. The attribute value can be derived from the field value of the specified fieldName. In the case of numerical it is straightforward i.e. attributeValue = numerical_field_value. In the case of freshness however, attributeValue = (time.now() - datetime_field_value).
interpolationType

enum (InterpolationType)

The interpolation type to be applied to connect the control points listed below.
controlPoints[]

object (ControlPoint)

The control points used to define the curve. The monotonic function (defined through the interpolationType above) passes through the control points listed here.

AttributeType

The attribute(or function) for which the custom ranking is to be applied.

Enums
ATTRIBUTE_TYPE_UNSPECIFIED Unspecified AttributeType.
NUMERICAL The value of the numerical field will be used to dynamically update the boost amount. In this case, the attributeValue (the x value) of the control point will be the actual value of the numerical field for which the boostAmount is specified.
FRESHNESS For the freshness use case the attribute value will be the duration between the current time and the date in the datetime field specified. The value must be formatted as an XSD dayTimeDuration value (a restricted subset of an ISO 8601 duration value). The pattern for this is: [nD][T[nH][nM][nS]]. For example, 5D, 3DT12H30M, T24H.

InterpolationType

The interpolation type to be applied. Default will be linear (Piecewise Linear).

Enums
INTERPOLATION_TYPE_UNSPECIFIED Interpolation type is unspecified. In this case, it defaults to Linear.
LINEAR Piecewise linear interpolation will be applied.

ControlPoint

The control points used to define the curve. The curve defined through these control points can only be monotonically increasing or decreasing(constant values are acceptable).

JSON representation 
{
  "attributeValue": string,
  "boostAmount": number
}
Fields
attributeValue

string

Can be one of: 1. The numerical field value. 2. The duration spec for freshness: The value must be formatted as an XSD dayTimeDuration value (a restricted subset of an ISO 8601 duration value). The pattern for this is: [nD][T[nH][nM][nS]].
boostAmount

number

The value between -1 to 1 by which to boost the score if the attributeValue evaluates to the value specified above.

QueryExpansionSpec

Specification to determine under which conditions query expansion should occur.

JSON representation 
{
  "condition": enum (Condition),
  "pinUnexpandedResults": boolean
}
Fields
condition

enum (Condition)

The condition under which query expansion should occur. Default to Condition.DISABLED.
pinUnexpandedResults

boolean

Whether to pin unexpanded results. If this field is set to true, unexpanded products are always at the top of the search results, followed by the expanded results.

Condition

Enum describing under which condition query expansion should occur.

Enums
CONDITION_UNSPECIFIED Unspecified query expansion condition. In this case, server behavior defaults to Condition.DISABLED.
DISABLED Disabled query expansion. Only the exact search query is used, even if SearchResponse.total_size is zero.
AUTO Automatic query expansion built by the Search API.

SpellCorrectionSpec

The specification for query spell correction.

JSON representation 
{
  "mode": enum (Mode)
}
Fields
mode

enum (Mode)

The mode under which spell correction replaces the original search query. Defaults to Mode.AUTO.

Mode

Enum describing under which mode spell correction should occur.

Enums
MODE_UNSPECIFIED Unspecified spell correction mode. In this case, server behavior defaults to Mode.AUTO.
SUGGESTION_ONLY Search API tries to find a spelling suggestion. If a suggestion is found, it is put in the SearchResponse.corrected_query. The spelling suggestion won't be used as the search query.
AUTO Automatic spell correction built by the Search API. Search will be based on the corrected query if found.

EmbeddingSpec

The specification that uses customized query embedding vector to do semantic document retrieval.

JSON representation 
{
  "embeddingVectors": [
    {
      object (EmbeddingVector)
    }
  ]
}
Fields
embeddingVectors[]

object (EmbeddingVector)

The embedding vector used for retrieval. Limit to 1.

EmbeddingVector

Embedding vector.

JSON representation 
{
  "fieldPath": string,
  "vector": [
    number
  ]
}
Fields
fieldPath

string

Embedding field path in schema.
vector[]

number

Query embedding vector.

NaturalLanguageQueryUnderstandingSpec

Specification to enable natural language understanding capabilities for search requests.

JSON representation 
{
  "filterExtractionCondition": enum (FilterExtractionCondition),
  "geoSearchQueryDetectionFieldNames": [
    string
  ]
}
Fields
filterExtractionCondition

enum (FilterExtractionCondition)

The condition under which filter extraction should occur. Default to [Condition.DISABLED][].
geoSearchQueryDetectionFieldNames[]

string

Field names used for location-based filtering, where geolocation filters are detected in natural language search queries. Only valid when the FilterExtractionCondition is set to ENABLED.

If this field is set, it overrides the field names set in ServingConfig.geo_search_query_detection_field_names.

FilterExtractionCondition

Enum describing under which condition filter extraction should occur.

Enums
CONDITION_UNSPECIFIED Server behavior defaults to [Condition.DISABLED][].
DISABLED Disables NL filter extraction.
ENABLED Enables NL filter extraction.

SearchAsYouTypeSpec

Specification for search as you type in search requests.

JSON representation 
{
  "condition": enum (Condition)
}
Fields
condition

enum (Condition)

The condition under which search as you type should occur. Default to Condition.DISABLED.

Condition

Enum describing under which condition search as you type should occur.

Enums
CONDITION_UNSPECIFIED Server behavior defaults to Condition.DISABLED.
DISABLED Disables Search As You Type.
ENABLED Enables Search As You Type.

SessionSpec

Session specification.

Multi-turn Search feature is currently at private GA stage. Please use v1alpha or v1beta version instead before we launch this feature to public GA. Or ask for allowlisting through Google Support team.

JSON representation 
{
  "queryId": string,
  "searchResultPersistenceCount": integer
}
Fields
queryId

string

If set, the search result gets stored to the "turn" specified by this query ID.

Example: Let's say the session looks like this: session { name: ".../sessions/xxx" turns { query { text: "What is foo?" queryId: ".../questions/yyy" } answer: "Foo is ..." } turns { query { text: "How about bar then?" queryId: ".../questions/zzz" } } }

The user can call /search API with a request like this:

session: ".../sessions/xxx" sessionSpec { queryId: ".../questions/zzz" }

Then, the API stores the search result, associated with the last turn. The stored search result can be used by a subsequent /answer API call (with the session ID and the query ID specified). Also, it is possible to call /search and /answer in parallel with the same session ID & query ID.
searchResultPersistenceCount

integer

The number of top search results to persist. The persisted search results can be used for the subsequent /answer api call.

This field is simliar to the summaryResultCount field in SearchRequest.ContentSearchSpec.SummarySpec.summary_result_count.

At most 10 results for documents mode, or 50 for chunks mode.

RelevanceThreshold

The relevance threshold of the search results. The higher relevance threshold is, the higher relevant results are shown and the less number of results are returned.

Enums
RELEVANCE_THRESHOLD_UNSPECIFIED Default value. In this case, server behavior defaults to Google defined threshold.
LOWEST Lowest relevance threshold.
LOW Low relevance threshold.
MEDIUM Medium relevance threshold.
HIGH High relevance threshold.

QuerySetSpec

Describes the specification of the query set.

JSON representation 
{
  "sampleQuerySet": string
}
Fields
sampleQuerySet

string

Required. The full resource name of the SampleQuerySet used for the evaluation, in the format of projects/{project}/locations/{location}/sampleQuerySets/{sampleQuerySet}.

QualityMetrics

Describes the metrics produced by the evaluation.

JSON representation 
{
  "docRecall": {
    object (TopkMetrics)
  },
  "docPrecision": {
    object (TopkMetrics)
  },
  "docNdcg": {
    object (TopkMetrics)
  },
  "pageRecall": {
    object (TopkMetrics)
  },
  "pageNdcg": {
    object (TopkMetrics)
  }
}
Fields
docRecall

object (TopkMetrics)

Recall per document, at various top-k cutoff levels.

Recall is the fraction of relevant documents retrieved out of all relevant documents.

Example (top-5): * For a single SampleQuery, If 3 out of 5 relevant documents are retrieved in the top-5, recall@5 = 3/5 = 0.6
docPrecision

object (TopkMetrics)

Precision per document, at various top-k cutoff levels.

Precision is the fraction of retrieved documents that are relevant.

Example (top-5): * For a single SampleQuery, If 4 out of 5 retrieved documents in the top-5 are relevant, precision@5 = 4/5 = 0.8
docNdcg

object (TopkMetrics)

Normalized discounted cumulative gain (NDCG) per document, at various top-k cutoff levels.

NDCG measures the ranking quality, giving higher relevance to top results.

Example (top-3): Suppose SampleQuery with three retrieved documents (D1, D2, D3) and binary relevance judgements (1 for relevant, 0 for not relevant):

Retrieved: [D3 (0), D1 (1), D2 (1)] Ideal: [D1 (1), D2 (1), D3 (0)]

Calculate NDCG@3 for each SampleQuery: * DCG@3: 0/log2(1+1) + 1/log2(2+1) + 1/log2(3+1) = 1.13 * Ideal DCG@3: 1/log2(1+1) + 1/log2(2+1) + 0/log2(3+1) = 1.63 * NDCG@3: 1.13/1.63 = 0.693
pageRecall

object (TopkMetrics)

Recall per page, at various top-k cutoff levels.

Recall is the fraction of relevant pages retrieved out of all relevant pages.

Example (top-5): * For a single SampleQuery, if 3 out of 5 relevant pages are retrieved in the top-5, recall@5 = 3/5 = 0.6
pageNdcg

object (TopkMetrics)

Normalized discounted cumulative gain (NDCG) per page, at various top-k cutoff levels.

NDCG measures the ranking quality, giving higher relevance to top results.

Example (top-3): Suppose SampleQuery with three retrieved pages (P1, P2, P3) and binary relevance judgements (1 for relevant, 0 for not relevant):

Retrieved: [P3 (0), P1 (1), P2 (1)] Ideal: [P1 (1), P2 (1), P3 (0)]

Calculate NDCG@3 for SampleQuery: * DCG@3: 0/log2(1+1) + 1/log2(2+1) + 1/log2(3+1) = 1.13 * Ideal DCG@3: 1/log2(1+1) + 1/log2(2+1) + 0/log2(3+1) = 1.63 * NDCG@3: 1.13/1.63 = 0.693

TopkMetrics

Stores the metric values at specific top-k levels.

JSON representation 
{
  "top1": number,
  "top3": number,
  "top5": number,
  "top10": number
}
Fields
top1

number

The top-1 value.
top3

number

The top-3 value.
top5

number

The top-5 value.
top10

number

The top-10 value.

State

Describes the state of an evaluation.

Enums
STATE_UNSPECIFIED The evaluation is unspecified.
PENDING The service is preparing to run the evaluation.
RUNNING The evaluation is in progress.
SUCCEEDED The evaluation completed successfully.
FAILED The evaluation failed.

Methods

create

 Creates a Evaluation.

get

 Gets a Evaluation.

list

 Gets a list of Evaluations.

listResults

 Gets a list of results for a given a Evaluation.