This page walks you through the steps to set up and start using Quality AI.
Prerequisites
The following steps ensure you can access Quality AI:
- Enable Conversational Insights for your project and ensure you have access to the Conversational Insights API.
- Enable Dialogflow runtime integration.
Use Quality AI
- Open Google Cloud CCAI and select your project where Conversational Insights is enabled.
- Click Insights
> insert_chart Quality AI.
Make a scorecard
You can create different scorecards for different business units.
- Click article Scorecards > + Add scorecard.
- Click Untitled Scorecard edit and add a name for your scorecard > Unspecified description edit, and add a description of what the scorecard is for.
Click + Add question > add a question to evaluate agent performance and an optional tag. For each question, you can select a tag: business, customer, or compliance.
Example
Question: Did the agent understand the customer's needs by asking thoughtful questions throughout the conversation?
Tag: Customer
Add instructions to define the interpretation of each answer choice.
Example
Instructions:
Yes: The agent asked thoughtful questions. OR The agent demonstrated active reading skills.
No: The agent did not ask thoughtful questions. The customer had to repeat themselves multiple times due to the agent's lack of understanding toward the customer's needs. The agent only asked necessary questions, such as their zip code or product name.
NA: Unable to ask questions. The interaction was for transfer or was a non-sales related interaction.
Select an answer type, enter the answer choices and their corresponding scores, and check the box to include
N/A, if applicable.Click + Add answer choice to include additional answer choices and their scores.
Example
Answer type: Yes/No
Answer choice and score: Yes 1, No 0
check_box Add 'N/A' (not available) as an answer choice. If selected, the question will not be included in the total score calculation.
Click Save.
Repeat steps 3-6 for each of your questions > Click Next.
Calibrate the AI model
You calibrate models using example conversations, which consist of conversations, associated questions, and expected answers. You must upload example conversations as a CSV file. For more details on formatting your example conversations, see the Quality AI best practices page.
Prepare example conversations
Quality AI provides a template which automatically generates the alphanumeric conversation, scorecard, and question IDs for a specific scorecard. You can also filter which conversations to include. You must add the answers for each question.
Follow these steps to create your example conversations template within the Quality AI console.
- Navigate to Conversations > click Create example conversations template.
- Fill in example details to illustrate how the template should look.
- Add filters to select specific conversations.
- Click Scorecard > select your scorecard.
- Add the answer for each question.
- Click Cloud Storage bucket and select a Cloud Storage bucket to store your example conversations.
Upload example conversations
When you have a CSV file with your example conversations, you must upload it to facilitate model calibration.
Within your Quality AI-enabled project, upload example conversations to your Cloud Storage bucket.
For a detailed walkthrough on how to use Google Cloud Storage with the Google Cloud console, see the Cloud Storage documentation.
In the Quality AI console, navigate to Scorecard > select your scorecard > click 2 Example Conversations.
Click + Add example conversations and add the path to your Cloud Storage bucket > Import.
Click Begin calibration to initiate model calibration, which can take from four to eight hours. (Time increases for scorecards with more questions or example conversations.)
After calibration, click Launch new version.