Retail is being renamed to Vertex AI Search for retail. We are in the process of updating content to reflect the new branding.

Conversational search

This page describes a Guided Search feature in Vertex AI Search for retail conversational search.

Conversational search enables retailers to provide a more interactive search experience for their users. The conversational search feature functions as part of the Guided Search package, benefiting customers by narrowing user queries and presenting relevant products faster.

How conversational search works

When it is enabled, Conversational Vertex AI Search for retail guides shoppers through their product search on merchandiser sites using conversation. After an initial text query in Vertex AI Search for retail the online shopper gets a relevant follow-up question and multiple choice options. The follow-up question can either be answered by the user in free text or by clicking on a conversational multiple choice option.

If conversational search is enabled on the retailer site, follow-up questions drive a conversation that ensues until one of the three following scenarios occur:

A preconfigured minimum product count is reached (a conversation is not useful when only two products show up, for instance).
The user clicks on a product and adds it to their cart (the objective).
Search and browse for retail runs out of AI-generated questions.

Under the hood

Conversational search is based on engaging the user with an ongoing conversation of multiple turns. Therefore, there is at least a second response required for conversational search to work. The user is presented with a follow-up question and suggested answers in the response. The user can respond to this follow-up question either by typing in their answer or by clicking on a suggested answer (multiple choice option).

Multiple choice The multiple choice option functions behind the scenes like a facet (an event type filter), which narrows the query using filtering. In the background, when the user clicks on a multiple choice response, a filter is applied to the query. Applying a filter using conversational multiple choice is identical to applying the same filter using dynamic facets or tiles.
Free text If the user responds in free text, a new and narrower query is generated. Learn more about how conversational search enriches filter and user event capturing in the user journey.

Benefits of conversational search

Augmenting the Vertex AI Search for retail experience with conversational search offers several benefits to both the retailer and the user.

Narrow queries in very few clicks

Conversational search offers a rapid way to filter 10,000 products down to less than 100 products more efficiently. This makes it more likely that the user decides to make a purchase, increasing the revenue-per-search rate.

Alternative to dynamic facets

Dynamic facets are associated with broad queries which have low revenue per query. Customers can become overwhelmed when they see 10,000s of results, creating the risk that they abandon their search experience. Particularly search queries that return high product counts have an unusually low revenue per query. Conversational search is able to refine queries and can be used in conjunction with dynamic facets. Conversational search offers some advantages over dynamic facets, being more human, more interactive, and using less on-page real estate.

Customizable generative questions adapted to retailer preferences

Conversational search encourages a human-in-the-loop interaction with the generative AI questions by allowing retailers to preliminarily edit, overwrite or deselect AI-generated questions according to their preferences, based on the uploaded catalog. Questions can be edited or disabled individually or in bulk in the Console Search for Retail or the API in order to tailor the questions they want to appear in the search.

Console - Admin experience

The console allows retailers to manage generative questions in a conversational Vertex AI Search for retail experience. Learn more about using generative questions in conversational search.

Steps to use the generative question service

Satisfy data requirements.
Configure manual overrides.
Turn the feature on.
Preview and test.

Data requirements

In the console, Conversational search and browse, under the COVERAGE CHECKS tab, or under Data Quality > CONVERSATION, you will see whether your search data is ready for conversational search.

In order to enable conversational search, you need to meet certain data requirements.

These are:

1000 queries per day: After you reach this first threshold, a conversation plan is generated that evaluate your inputs and outputs:
- Inputs: filter count in events
- Outputs: conversational coverage
25% conversational coverage: Calculated by Vertex AI Search for retail models, conversational coverage means the percentage of queries that have one question. A frequency-weighted 25% (by volume) of queries should have at least a first question that matches it.

If you don't have 25% conversational coverage yet, but have the first prerequisite 1000 queries per day, blocking and advisory checks begin to be applied to your outputs and inputs, respectively. Here, Vertex AI Search for retail begins to calculate by how much of a percentage your user-event-applied filters have to increase in order to reach the 25% conversational coverage threshold. The more filters that are uploaded, the higher the coverage reached.

To view your conversational readiness:

Go to the Conversation tab in the Data Quality page in the Search for Retail console. Here you will see the critical check of whether a minimum of 25% of search queries have at least one follow-up question, as well as advisory checks as to which percentage of user events with valid filters is needed to reach that conversational coverage goal. If you pass the critical check, with sufficient user events with valid filters, you can proceed to turn on conversational search.

Generative question controls

The generative AI writes a question for every indexable attribute in the catalog, using both names and values of attributes for system and custom attributes. These questions are generated by an LLM and aim to enhance the search experience. For example, for furniture type, values can be indoor or outdoor, the AI will synthesize a question about what type of furniture you are looking for.

Each facet will have one generated question. Based on historic user events and facet engagement from past search event data, the questions are sorted by expected frequency of the question appearing. The AI will first look at the questions on top, then find what is relevant by attribute. The list of questions is generated once. If a new attribute is added, it will be reflected in the list in two hours.

To control how generative questions are served, go to the Conversational search and browse page in the Search for Retail console.

Go to the Conversational Search and Browse page in the Search for Retail console.

Go to the Conversational Search and Browse page
Under the Manage AI Generated Questions tab, you can see all questions sorted by how often they are used, in query-weighted frequency, meaning how often they are served with common queries. The ranking uses the frequency field in the GenerativeQuestionConfig. This field is responsible for sorting the AI-generated questions by how often they are used.
You can use the filter option to filter the questions.
Check the box to enable question visibility for each attribute.
Click the pencil icon at the end of each row to open an edit panel for each question.

You can also make bulk edits by:

Checking or unchecking the boxes next to the questions you want to include or exclude in conversation.
Decide which action you want to take for the multiple questions by clicking either the ALLOW IN CONVERSATION or the DISALLOW IN CONVERSATION buttons that appear at the top of the list.

Alt text

How to use generative questions in conversational search

The generative question service API provides controls to mitigate potential inconsistencies in the LLM output. These can be managed from the console. Here, retailers can also configure conversational search by toggling its enabled state and setting the minimum number of products required to trigger it.

You can define the questions, specifying the question itself, potential answers, and whether the question is allowed in the conversation. Individual questions can be generated by an LLM or overridden by the retailer. The console supports reviewing AI-generated questions, allowing retailers to override them or toggle their conversational status. Questions can also be bulk edited.

Edit individual questions

You can also use controls to curate the individual questions. It is recommended to do this before you turn conversational search on.

For each question, there are two options. Click the pencil icon in the last column to access the questions visible to the users panel:

Turn off a question for all queries: The question will be enabled by default. Clear (or check again) the box next to Allowed in conversation. This option skips the question altogether. A retailer may opt to disable a question entirely if it does not relate to the queried attributes or could be misconstrued as inappropriate in some way (a question such as "What dress size are you looking for?" may be perceived as prying about a shopper's weight.)
Rewrite a question: In the fly-out panel, you can see the AI-generated question, what attribute it is attached to and what values the attribute has. Click the pencil to rewrite it.

Turn on conversational search

Once you have edited your generative AI questions in the console, you are ready to turn on conversational search.

To enable conversational search, go to the Conversational search and browse page in the Search for Retail console.

Go to the Conversational Search and Browse page in the Search for Retail console.

Go to the Conversational Search and Browse page
Under the CONFIGURE tab in the Search for Retail, you will find the system-wide setting. This includes setting the minimum products needed to match the query before a conversation can happen, thus when questions are generated. This minimum number is =>2. The minimum can be configured to be higher but never lower than 2. Consider the amount of products in your catalog you want to have returned in the search for users to begin a conversation. For example, a sweet spot for this number is one row to a page for minimum search results to trigger a conversation.
Switch the toggle to on. This page also provides information as to the status of your blocking and advisory checks. If you have enough search queries with at least one follow-up question, your site is now conversational search enabled.

Evaluate and test

Evaluate lets you preview the serving experience by running a test search and testing your questions against displayed facets. This part of the console provides you with a preview of your serving experience with conversational search.

To do so, find this module under the SEARCH or BROWSE in the Evaluate page of the Search for Retail console.

Go to the Evaluate page in the Search for Retail console.

Go to the Evaluate page
In the Search Evaluation field, enter a test query that makes sense based on the catalog you have uploaded to search. Click Search Preview. You will see search results and if you have conversational search enabled, you will see generative questions on the right panel.
On the right panel, you will see a list of test questions.

Generative question API - Admin experience

This section describes how to use the generative question API to integrate the conversational search API into your UI, manage the generative questions, and serve the feature on your site.

API integration

Manage the GenerativeQuestion API from the Search for Retail using:

Objects:

GenerativeQuestionsFeatureConfig
GenerativeQuestionConfig
GenerativeQuestions Service
- UpdateGenerativeQuestionsFeatureConfiguration
- UpdateGenerativeQuestionConfig
- ListGenerativeQuestionConfigs
- GetGenerativeQuestionFeatureConfig
- BatchUpdateGenerativeQuestionConfigs

The core of the integrating this feature lies in defining the "question" resource. This includes the question itself and whether the question is allowed in the conversation. The question is by default generated by an LLM but can be overridden by the administrator.

Enable the feature

Object:

GenerativeQuestionsFeatureConfig

This object is a control configuration file for enabling the feature for generative questions to manage the overall serving experience of conversational search. GenerativeQuestionsFeatureConfig obtains using a GET method attribute information and whether the attributes are indexable or not from the catalog associated with the project.

The feature_enabled switch determines whether questions are used at serving time. It manages the top-level toggles in the console.

Learn more about how you can turn on conversational search in the console.

Enabling the feature in the API

// Configuration for overall generative question feature state.
message GenerativeQuestionsFeatureConfig {
...

  // Resource name of the affected catalog.
  // Format: projects/{project}/locations/{location}/catalogs/{catalog}
  string catalog = 1 [(google.api.field_behavior) = REQUIRED];

  // Determines whether questions will be used at serving time.
  // Note: This feature cannot be enabled until initial data requirements are
  // satisfied.
  bool feature_enabled = 2 [(google.api.field_behavior) = OPTIONAL];

  // Minimum number of products in the response to trigger follow-up questions.
  // Value must be 0 or positive.
  int32 minimum_products = 3 [(google.api.field_behavior) = OPTIONAL];
}

Conversation-relevant fields
catalog	string The resource name of the affected catalog at the project level. Format: projects/{project}/locations/{location}/catalogs/{catalog}.
feature_enabled	boolean Switches questions on or off. Optional field. It cannot be enabled until data requirements are met. If attempted, it throws an error.
minimum_products	Integer32 Minimum number of products in the response to trigger follow-up questions. Must have a value of 0 or greater.

Manage the generative questions

Object:

GenerativeQuestionConfig

It can be conversation-enabled with the boolean field allowed_in_conversation. It controls the configuration for a single generated question.

Fields (control behaviors for conversation highlighted)
catalog	string Used to identify which set of attributes (and by extension questions) are available. These values are all defined in the catalog. Required field.
facet	string Facet to which a question is associated. Required field.
generated_question	string The LLM generated default question. Output only.
final_question	string The question that will be asked. It can have a max length of 300 bytes. Optional field.
example_values	Repeated string Values that can be used to answer the question. Output only
frequency	float The ratio of how often a question was asked. Output only.
allowed_in_conversation	boolean Whether the question is asked at serving time. This field is optional.

Serving experience enabled by this feature

service GenerativeQuestionService {
...
}

The generative questions service is used for managing LLM-generated questions. Its parent object is the catalog, where it retrieves information to return questions for a given catalog. The service is used to manage the overall generative question feature state, make individual or batch changes, and toggle questions on or off. Data requirements must be met to interface with the Service API and the questions need to first be initialized before they can be managed.

The service interacts with the feature level and question level configs with two sets of handlers:

GenerativeQuestionsFeatureConfig handlers (feature-level):
1. Update: lets you change minimum products and enable fields
2. Get: returns an object
GenerativeQuestion Config handlers (question-level):
1. List: Returns all questions for a given catalog
2. Update: Individual question management
3. Batch Update: Grouped question management

The service will return a semantically appropriate question based on the initial query.

A follow-up question is generated by the LLM model and can be overridden. The questions are displayed based on how likely it will be used by customers by calling the search event history. If there is no search event history, the fallback is on the retailer search logs.

Different questions are generated based on the previous query. There are no fixed weights. The AI that drives the LLM-generated questions learns from the queries, and changes the weighting for every query, so that "shirt", for example, weighs the category very heavily, but "XL red shirt" weighs category, size and color.

Conversational search configuration API - Serving experience

The conversational search configuration API is integrated with the Vertex AI API search API.

API integration

The Configuration API for the conversational search feature sits on top of the existing Search API. To support the new feature, conversational search, the following changes have been made to the already existing Vertex AI Search for retail main (query) API:

ConversationalSearchSpec: This optional field has been added in SearchRequest but is required if you want to use the conversational search feature. The field reuses the SearchRequest fields, query and filter. It also includes a field to enable a follow-up question served to the user after an initial query and a conversation_id to maintain the state of the conversation between the client and server.
ConversationalSearchResult: A proto contains extra information needed to be returned for the conversational CRS flow in SearchResponse. This includes a conversation_id, refined_query, additional_filters, follow_up_question, and suggested_answers (see the User journey section).

User journey

The conversational flow works as follows: the user initiates a search with an initial query and the followup_conversation_requested flag set to "TRUE." The user then selects an answer or provides free-text input, which is sent back to the API using the user_answer field. The API then refines the search results based on the user's input and provides a new follow-up question, prompting a follow-up query and continuing the conversation in multiple turns until the user finds what they're looking for on the retailer website.

Assuming Conversational search is enabled on the website, the user journey and subsequent interaction with Vertex AI Search for retail follows this path:

Step 1. First query comes from user
Step 1a. Follow-up conversation requested sent to search
Step 1b. Initial Search response with refined query and suggested answers
Scenario 2: User selects multiple choice
Step 2a. Selected answer filter sent to search
Step 2b. Search run again with filter applied
Scenario 3: User selects free text
Step 3a. Text answer sent to Search
Step 3b. Search run again with a modified query

Step 1. First query comes from user

conversational_search_spec: The introduction of a ConversationalSearchSpec field within the SearchRequest message allows the system to distinguish between conversational and regular searches. This determination influences whether users receive additional conversational responses, thus preserving the original search capabilities while extending it for conversational interactions. The conversational_search_spec field is in the message format and houses details necessary for the conversational flow, such as user answers, conversation IDs and whether the user wants a follow up conversation. This information guides the system in understanding the context and user interactions.

If the followup_conversation_requested boolean If this field is set to TRUE, the API responds with an initial set of results and a follow-up question. The user will be provided a conversational experience in their search. If this field is set to "FALSE", no follow-up question is served.

Step 1a. Retailer → search: Initial query with conversation enabled

JSON Request

{
  query: "dress",
  conversational_search_spec {
   followup_conversation_requested: true
  }
}

Step 1b. Search → retailer: conversation ID, refined query, follow-up question, suggested answers

JSON Response

{
  conversational_search_result: {
    conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
    refined_query: "dress"
    followup_question: "What is the color?"
    suggested_answers {
      product_attribute_value {
        name: "colors",
        value: "yellow"
      }
    }
    suggested_answers {
      product_attribute_value {
        name: "colors",
        value: "blue"
      }
    }
    suggested_answers {
      product_attribute_value {
        name: "colors",
        value: "red"
      }
    }
  }
}

conversational_search_result: ConversationalSearchResult is a proto that contains extra information returned from the conversational flow. The Conversational_search_result field in SearchResponse provides supplementary data that is crucial for UI rendering on the client side. The conversational_search_result field uses nested messages such as SuggestedAnswer and AdditionalFilter to structure the returned data in a clear and comprehensible manner. This hierarchical structure ensures that the information exchanged between the client and server is well-defined and interpretable.
The suggested answers are sorted by which facet values in the catalog were most often engaged in the user events as filters for all queries. The sorting uses the engagement_rate field in the follow-up question generated by the ConversationalSearch proto. This field captures the historical engagement rate that the filter shows from the retailer's search events sent to the server.

What to do with the initial conversational search return

conversation_id: a unique identifier for the ongoing search session, which must be stored and used in subsequent requests. It is empty on the initial query, and the server will reply with the conversation_id. The field is used to keep track of "which conversation is going on" - because one shopper may have multiple tabs or devices open.
refined_query: tracks the current query and is updated as the user interacts with Search. This changes if a shopper chooses free text input. If the shopper selects a multiple choice option, this field will remain the same.
followup_question: tells the retailer what question to show to the user. It contains the fields question, suggested_answers
suggested_answers: contains product attribute values containing name and value key-value pairs for each suggested answer. It provides a list of multiple-choice options. This field can be set to just show the first N multiple-choice results. These are sorted by an engagement_rate field, which represents the probability that the question will be involved in a search.

Scenario 2: User selects a multiple choice option

If a user selected a multiple choice answer yellow:

The conversation_id is restored from session storage.
followup_conversation_requested is set to be true.
user_answer string, uses either "selected_answer," which contains one product_attribute_value key-value pair, or text_answer which contains the value free text input to indicate the user's choice. This field is within the conversational_search_spec field and contains further nested messages like "SelectedAnswer" to specify the user input types (text or selected answers).
The result reverts to calling the SearchResults object and its fields.
selected_answerThis field passes the product attributes to guide the conversational search.
The filter is reused from SearchRequest and is added by the client.

Step 2a. Retailer → search: selected answer filter

JSON Request

{
  query: "dress"
  filter: "colors:ANY(\"yellow\")"
  conversational_search_spec {
    followup_conversation_requested: true
    conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
    user_answer {
      selected_answer {
        product_attribute_value {
          name: "colors"
          value: "yellow"
        }
      }
    }
  }
}

Step 2b. Search → retailer: filters applied

JSON Response

{
  conversational_search_result {
    conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
    refined_query: "dress"
    additional_filter {
      product_attribute_value {
        name: "colors",
        value: "yellow"
      }
    }
    followup_question: "What is the occasion?"
    suggested_answers {
      product_attribute_value {
        name: "attribute.occasion",
        value: "wedding"
      }
    }
    suggested_answers {
      product_attribute_value {
        name: "attribute.occasion",
        value: "party"
      }
    }
  }
}

What to do with a multiple choice response

The search response is basically identical to the response to the first query, except it also uses an additional_filter. This field is used to "check off" the filter box for color = "yellow" and to add to any other filters that the user has already selected. The additional_filter is added to the event (in the filter field) if there is not a filter in the current request. The client should check if the filter is already in their current applied filters. If not, this additional field should be added to their current applied filters for all following requests.

Scenario 3: User selects a free text input

If a user types in lavender:

the conversation_id is restored from session storage
followup_conversation_requested is set to true
user_answer is set for what the user inputs (with the text_answer: prefix)

Step 3a. Retailer → search: text answer

JSON Request

{
  query: "dress"
  conversational_search_spec {
    followup_conversation_requested: true
    conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
    user_answer: {
        text_answer: "lavender"
    }
  }
}

Note the absence of a filter field in the free text request.

Step 3b. Search → retailer: run with modified query

JSON Response

{
  conversational_search_result {
    conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
    refined_query: "lavender dress"
    followup_question: "What is the occasion?"
    suggested_answers {
      product_attribute_value {
        name: "attribute.occasion"
        value: "wedding"
      }
    }
    suggested_answers {
      product_attribute_value {
        name: "attribute.occasion"
        value: "party"
      }
    }
  }
}

What to do with a free text response

Similar to the initial response, but notice that the refined_query is now updated accordingly. Notice that the user facing query has changed from "dress" to "lavender dress".
If the user gives a free text answer that matches a suggested answer, this case will be treated like a multiple choice response (see above).
Send the correct refined query in the events.