Conversational filtering developer's guide

The following is a developer's guide on how to integrate conversational product filtering in your API.

Administrator experience

Manage the generative questions and conversational product filtering directly in the API, or in the Search for commerce console, and set it up in the Data quality and Evaluate sections of the Search for commerce console.

// Configuration for overall generative question feature state.
message GenerativeQuestionsFeatureConfig {
...

// Resource name of the affected catalog.
// Format: projects/{project}/locations/{location}/catalogs/{catalog}
string catalog = 1 [(google.api.field_behavior) = REQUIRED];

// Determines whether questions are used at serving time.
// Note: You cannot enable this until initial data requirements are
// satisfied.
bool feature_enabled = 2 [(google.api.field_behavior) = OPTIONAL];

// Minimum number of products in the response to trigger follow-up questions.
// Value must be 0 or positive.
int32 minimum_products = 3 [(google.api.field_behavior) = OPTIONAL];
}

Serving experience

Conversational product filtering is based on engaging your user with an ongoing conversation of multiple turns. Therefore, there's at least a second response required for conversational product filtering to work. Your user is presented with a follow-up question and suggested answers in the response, and your user can respond to this follow-up question either by either entering their answer or by clicking on a suggested answer (multiple choice option).

• Multiple choice The multiple choice option functions behind the scenes like a facet (an event type filter), which narrows the query using filtering. In the background, when the user clicks on a multiple choice response, a filter is applied to the query. Applying a filter using conversational multiple choice is identical to applying the same filter using dynamic facets or tiles.

• Free text If the user responds in free text, a new and narrower query is generated. Learn more about how conversational product filtering enriches filter and user event capturing at the API level.

Service enabled by the feature

The generative questions service (service GenerativeQuestionService{...}) is used for managing LLM-generated questions. Its parent object is the catalog, where it retrieves information from to return questions for a given catalog. The service is used to manage the overall generative question feature state, make individual or batch changes, and toggle questions on or off. Data requirements must be met to interface with the Service API, and the questions must be initialized before the questions can be managed.

The service interacts with the feature level and question level configuration files with two sets of handlers:

• GenerativeQuestionsFeatureConfig handlers (feature-level):

  1. Update Lets you change minimum products and enable fields.
  2. Get Returns an object.

• GenerativeQuestion Config handlers (question-level):

  1. List Returns all questions for a given catalog.
  2. Update Performs individual question management.
  3. Batch Update Performs grouped question management.

The service returns a semantically appropriate question based on the initial query.

A follow-up question is generated by the LLM model and can be overridden. The questions are displayed based on how likely it is used by customers by calling the search event history. If there is no search event history, the fallback is on the commerce search logs.

Different questions are generated based on the previous query. There are no fixed weights. The AI that drives the LLM-generated questions learns from the queries and changes the weighting for every query, so that "shirt", for example, weighs the category very heavily, but "XL red shirt" weighs category, size and color.

Configure the serving experience

Configure the serving experience by integrating the conversational filtering configuration API with the Search API.

User journey in the API

The conversational flow works as follows: The user initiates a search with an initial query and sets the mode flag to CONVERSATIONAL_FILTER_ONLY. The user then selects an answer or provides free-text input, which is sent back to the API using the user_answer field.

The Conversational API provides the additional_filter field in the response. The user must apply these filters to the Search API follow-up request. The search results are based on the user's input and provide a new follow-up question, prompting a follow-up query and continuing the conversation in multiple turns until the user finds what they're looking for on the retailer website.

User scenarios

Assuming conversational product filtering is enabled on the website, the user journey and subsequent interaction with Vertex AI Search for commerce follows this path:

• Step 1. First query comes from user to both Search and Conversational API. The Search API only returns search results. The Conversational API returns the suggested answers and follow-up questions. Call the Search API for the same query or page_category and fetch the search results.
• Step 2. Follow-up conversation requested is sent to conversational search. Call the Conversational API with the right conversation filtering mode.
• Step 3. Initial search response with search results only. The Conversational API refines the query by returning the suggested answers and follow-up questions.
• Scenario A: User selects multiple choice.
  • Step 1A. Selected answer filter sent to the Conversational API.
  • Step 2A. Both Conversation and Search APIs run with applied filter.
• Scenario B: User selects free text.
  • Step 1B. Text answer sent to the Conversational API. Use the Conversational API to send the user answer.
  • Step 2B. The user gets a conversational follow-up question with some suggested answers in the Conversational response. Search run again with a modified query. Conversational API sends another question and additional_filter. This filter must be applied to the search results fetched from the Search API in the first step.

Step 1. First query comes from user

The conversationalFilteringMode mode in the Conversational API is option and the default DISABLED can be set to ENABLED to control conversational filtering.

First, developers need to create the following search request by setting the product or item as the query, in this example, "dress":

{
  query: "dress",
}

Additional actions on the client side to enable conversationally filtered searches:

• Developers must also create a conversational search request by setting "dress" as the query.

• Developers must set mode to CONVERSATIONAL_FILTER_ONLY in order to get a conversational response. Otherwise, if it's set to DISABLED, no follow-up question is supplied.

Step 2. Retailer → search: Initial query with conversation enabled

{
  "query": "dress",
  "conversational_search_spec":  {
    "mode": "CONVERSATIONAL_FILTER_ONLY"
  }
}

Step 3. Search → retailer: conversation ID, refined query, follow-up question, suggested answers

{
  conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
  refined_query: "dress"
  conversational_filtering_result: {
    followup_question{
      followup_question: "What is the color?"
      suggested_answers {
        product_attribute_value {
          name: "colors",
          value: "yellow"
        }
      }
    suggested_answers {
      product_attribute_value {
        name: "colors",
        value: "blue"
      }
    }
    suggested_answers {
      product_attribute_value {
        name: "colors",
        value: "red"
      }
    }
   }
  }
}

Conversational product filtering serves these options for continued conversational engagement, leading to faster search refinement:

Scenario A: User selects a multiple choice option

If a user selected a multiple choice answer yellow:

• Developers must restore the conversation_id from session storage.
• Set mode to be CONVERSATIONAL_FILTER_ONLY.
• Set user_answer for what user selects.

Step 1A. Retailer → search: selected answer filter

{
  query: "dress"
  conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
  conversational_search_spec {
    "mode": "CONVERSATIONAL_FILTER_ONLY"
    user_answer {
      selected_answer {
        product_attribute_value {
          name: "colors"
          value: "yellow"
        }
      }
    }
  }
}

Step 2A. Search → retailer: filters applied

{
  conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
  refined_query: "dress"
  conversational_filtering_result {
    additional_filter {
      product_attribute_value {
        name: "colors",
        value: "yellow"
      }
    }
    followup_question{
    followup_question: "What is the occasion?"
    suggested_answers {
      product_attribute_value {
        name: "attribute.occasion",
        value: "wedding"
      }
    }
    suggested_answers {
      product_attribute_value {
        name: "attribute.occasion",
        value: "party"
      }
    }
  }
 }
}

Scenario B: User selects a free text input

If a user types in lavender:

• Developers should restore the conversation_id from session storage.
• Set followup_conversation_requested to be true.
• Set user_answer for what user inputs (with the "text_answer:" prefix).

Step 1B. Retailer → search: text answer

{
  query: "dress"
  conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
  conversational_search_spec {
    "mode": "CONVERSATIONAL_FILTER_ONLY"
    user_answer: {
        text_answer: "lavender"
    }
  }
}

Step 2B. Search → retailer: run with modified query

{
  conversation_id: "c154d073-87f5-4edb-accd-587eabe014ff"
  refined_query: "lavender dress"
  conversational_filtering_result {
    Followup_question {
    followup_question: "What is the occasion?"
    suggested_answers {
      product_attribute_value {
        name: "attribute.occasion"
        value: "wedding"
      }
    }
    suggested_answers {
      product_attribute_value {
        name: "attribute.occasion"
        value: "party"
      }
    }
  }
 }
}

Iterative improvement with testing

Conversational product filtering is an ongoing process of optimization, requiring continuous refinement and data-driven decisions. The goal is to maximize feature effectiveness by understanding shopper behavior and adapting the design accordingly.

Influenced by various factors such as market trends, competitor offerings, and changes in personal preferences, shopper behaviors are dynamic and evolve over time. It's important to continue to experiment and iterate on your design, and test new approaches as you gather more data and observe how shoppers interact with the AI features. This ongoing cycle of experimentation, data analysis, and refinement ensures that the AI features remain relevant, effective, and optimized for your evolving user base.

Regularly review performance metrics, conduct user surveys, and analyze feedback to identify areas for improvement and new opportunities for innovation. This commitment to continuous iteration is key to long-term success in AI feature deployment.

Lessons learned

These are the lessons learned after successive testing:

• Experiment continuously: The optimal result is often not the first design you try.
• Iterate and adapt: User behaviors evolve. Continue to iterate on your designs and test new approaches as you gather more data and observe how shoppers interact with the feature.
• Beyond A/B: Don't limit yourself to just A/B testing (comparing two versions). Instead, conduct many A/B/C/D/E/F tests to explore a wider range of UI designs and placement options.

Key metrics for optimization

To effectively optimize Vertex AI Search for commerce, it's crucial to define and track relevant metrics that provide insights into user engagement, satisfaction, and the overall impact of the features. Key metrics to consider include:

• Conversion rate: The percentage of users who complete the targeted action, such as making a purchase.
• User satisfaction scores (such as NPS, CSAT): Direct feedback from users on their experience with the AI feature, providing qualitative insights into usability and perceived value.
• Adoption rate: The percentage of shoppers that actively use conversational product filtering, indicating its visibility and perceived utility.