Get started with Quality AI

This page walks you through the steps to set up and start using Quality AI.

Prerequisites

The following steps ensure you can access Quality AI:

  1. Enable Conversational Insights for your project and ensure you have access to the Conversational Insights API.
  2. Enable Dialogflow runtime integration.

Use Quality AI

  1. Open Google Cloud CCAI and select your project where Conversational Insights is enabled.
  2. Click Insights > insert_chart Quality AI.

Make a scorecard

You can create different scorecards for different business units.

  1. Click article Scorecards > + Add scorecard.
  2. Click Untitled Scorecard edit and add a name for your scorecard > Unspecified description edit, and add a description of what the scorecard is for.

  3. Click + Add question > add a question to evaluate agent performance and an optional tag. For each question, you can select a tag: business, customer, or compliance.

    Example

    Question: Did the agent understand the customer's needs by asking thoughtful questions throughout the conversation?

    Tag: Customer

  4. Add instructions to define the interpretation of each answer choice.

    Example

    Instructions:

    Yes: The agent asked thoughtful questions. OR The agent demonstrated active reading skills.

    No: The agent did not ask thoughtful questions. The customer had to repeat themselves multiple times due to the agent's lack of understanding toward the customer's needs. The agent only asked necessary questions, such as their zip code or product name.

    NA: Unable to ask questions. The interaction was for transfer or was a non-sales related interaction.

  5. Select an answer type, enter the answer choices and their corresponding scores, and check the box to include N/A, if applicable.

  6. Click + Add answer choice to include additional answer choices and their scores.

    Example

    Answer type: Yes/No

    Answer choice and score: Yes 1, No 0

    check_box Add 'N/A' (not available) as an answer choice. If selected, the question will not be included in the total score calculation.

  7. Click Save.

  8. Repeat steps 3-6 for each of your questions > Click Next.

Calibrate the AI model

You calibrate models using example conversations, which consist of conversations, associated questions, and expected answers. You must upload example conversations as a CSV file. For more details on formatting your example conversations, see the Quality AI best practices page.

Prepare example conversations

Quality AI provides a template which automatically generates the alphanumeric conversation, scorecard, and question IDs for a specific scorecard. You can also filter which conversations to include. You must add the answers for each question.

Follow these steps to create your template for example conversations within the Quality AI console.

  1. Navigate to Conversations and add filters to select specific conversations.
  2. Click Create example conversations template.
  3. Click Scorecard and select the name of your scorecard.
  4. Click Cloud Storage Destination and enter the location of a file in your Cloud Storage bucket.

Upload example conversations

When you have a CSV file with your example conversations, you must upload it to facilitate model calibration.

  1. Within your Quality AI-enabled project, upload example conversations to your Cloud Storage bucket.

    For a detailed walkthrough on how to use Google Cloud Storage with the Google Cloud console, see the Cloud Storage documentation.

  2. In the Quality AI console, navigate to Scorecards > select your scorecard > click Next to select example conversations.

  3. Click + Add example conversations and enter the path to your Cloud Storage bucket, then click Add.

  4. Click Begin calibration to initiate model calibration, which can take from four to eight hours. (Time increases for scorecards with more questions or example conversations.)

  5. After calibration, click Launch new version.

What's next?