Resource: QaQuestion
A single question to be scored by the Insights QA feature.
JSON representation |
---|
{ "name": string, "abbreviation": string, "createTime": string, "updateTime": string, "questionBody": string, "answerInstructions": string, "answerChoices": [ { object ( |
Fields | |
---|---|
name |
Identifier. The resource name of the question. Format: projects/{project}/locations/{location}/qaScorecards/{qaScorecard}/revisions/{revision}/qaQuestions/{qaQuestion} |
abbreviation |
Short, descriptive string, used in the UI where it's not practical to display the full question body. E.g., "Greeting". |
createTime |
Output only. The time at which this question was created. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
updateTime |
Output only. The most recent time at which the question was updated. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
questionBody |
Question text. E.g., "Did the agent greet the customer?" |
answerInstructions |
Instructions describing how to determine the answer. |
answerChoices[] |
A list of valid answers to the question, which the LLM must choose from. |
tags[] |
User-defined list of arbitrary tags for the question. Used for grouping/organization and for weighting the score of each question. |
order |
Defines the order of the question within its parent scorecard revision. |
metrics |
Metrics of the underlying tuned LLM over a holdout/test set while fine tuning the underlying LLM for the given question. This field will only be populated if and only if the question is part of a scorecard revision that has been tuned. |
tuningMetadata |
Metadata about the tuning operation for the question.This field will only be populated if and only if the question is part of a scorecard revision that has been tuned. |
AnswerChoice
Message representing a possible answer to the question.
JSON representation |
---|
{ "key": string, // Union field |
Fields | |
---|---|
key |
A short string used as an identifier. |
Union field value . The answer value may be one of a few different types. value can be only one of the following: |
|
strValue |
String value. |
numValue |
Numerical value. |
boolValue |
Boolean value. |
naValue |
A value of "Not Applicable (N/A)". If provided, this field may only be set to |
score |
Numerical score of the answer, used for generating the overall score of a QaScorecardResult. If the answer uses naValue, this field is unused. |
Metrics
A wrapper representing metrics calculated against a test-set on a LLM that was fine tuned for this question.
JSON representation |
---|
{ "accuracy": number } |
Fields | |
---|---|
accuracy |
Output only. Accuracy of the model. Measures the percentage of correct answers the model gave on the test set. |
TuningMetadata
Metadata about the tuning operation for the question. Will only be set if a scorecard containing this question has been tuned.
JSON representation |
---|
{
"totalValidLabelCount": string,
"datasetValidationWarnings": [
enum ( |
Fields | |
---|---|
totalValidLabelCount |
Total number of valid labels provided for the question at the time of tuining. |
datasetValidationWarnings[] |
A list of any applicable data validation warnings about the question's feedback labels. |
tuningError |
Error status of the tuning operation for the question. Will only be set if the tuning operation failed. |
Methods |
|
---|---|
|
Create a QaQuestion. |
|
Deletes a QaQuestion. |
|
Gets a QaQuestion. |
|
Lists QaQuestions. |
|
Updates a QaQuestion. |