Send feedback
Class InferenceParameter (2.41.2)
Stay organized with collections
Save and categorize content based on your preferences.
Version latestkeyboard_arrow_down
InferenceParameter ( mapping = None , * , ignore_unknown_fields = False , ** kwargs )
Attributes
Name
Description
max_output_tokens
int
Optional. Maximum number of the output tokens
for the generator.
This field is a member of oneof
_ _max_output_tokens
.
temperature
float
Optional. Controls the randomness of LLM
predictions. Low temperature = less random. High
temperature = more random. If unset (or 0), uses
a default value of 0.
This field is a member of oneof
_ _temperature
.
top_k
int
Optional. Top-k changes how the model selects tokens for
output. A top-k of 1 means the selected token is the most
probable among all tokens in the model's vocabulary (also
called greedy decoding), while a top-k of 3 means that the
next token is selected from among the 3 most probable tokens
(using temperature). For each token selection step, the top
K tokens with the highest probabilities are sampled. Then
tokens are further filtered based on topP with the final
token selected using temperature sampling. Specify a lower
value for less random responses and a higher value for more
random responses. Acceptable value is [1, 40], default to
40.
This field is a member of oneof
_ _top_k
.
top_p
float
Optional. Top-p changes how the model selects tokens for
output. Tokens are selected from most K (see topK parameter)
probable to least until the sum of their probabilities
equals the top-p value. For example, if tokens A, B, and C
have a probability of 0.3, 0.2, and 0.1 and the top-p value
is 0.5, then the model will select either A or B as the next
token (using temperature) and doesn't consider C. The
default top-p value is 0.95. Specify a lower value for less
random responses and a higher value for more random
responses. Acceptable value is [0.0, 1.0], default to 0.95.
This field is a member of oneof
_ _top_p
.
Send feedback
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License , and code samples are licensed under the Apache 2.0 License . For details, see the Google Developers Site Policies . Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-08-28 UTC.
Need to tell us more?
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-28 UTC."],[],[],null,["# Class InferenceParameter (2.41.2)\n\nVersion latestkeyboard_arrow_down\n\n- [2.41.2 (latest)](/python/docs/reference/dialogflow/latest/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.41.1](/python/docs/reference/dialogflow/2.41.1/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.40.0](/python/docs/reference/dialogflow/2.40.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.39.1](/python/docs/reference/dialogflow/2.39.1/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.38.0](/python/docs/reference/dialogflow/2.38.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.37.0](/python/docs/reference/dialogflow/2.37.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.36.0](/python/docs/reference/dialogflow/2.36.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.35.0](/python/docs/reference/dialogflow/2.35.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.34.0](/python/docs/reference/dialogflow/2.34.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.33.0](/python/docs/reference/dialogflow/2.33.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.32.0](/python/docs/reference/dialogflow/2.32.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.30.2](/python/docs/reference/dialogflow/2.30.2/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.29.0](/python/docs/reference/dialogflow/2.29.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.28.3](/python/docs/reference/dialogflow/2.28.3/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.27.0](/python/docs/reference/dialogflow/2.27.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.26.0](/python/docs/reference/dialogflow/2.26.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.25.0](/python/docs/reference/dialogflow/2.25.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.24.1](/python/docs/reference/dialogflow/2.24.1/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.23.3](/python/docs/reference/dialogflow/2.23.3/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.22.0](/python/docs/reference/dialogflow/2.22.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.21.0](/python/docs/reference/dialogflow/2.21.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.20.0](/python/docs/reference/dialogflow/2.20.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.19.1](/python/docs/reference/dialogflow/2.19.1/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.18.0](/python/docs/reference/dialogflow/2.18.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.17.0](/python/docs/reference/dialogflow/2.17.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.16.1](/python/docs/reference/dialogflow/2.16.1/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.15.2](/python/docs/reference/dialogflow/2.15.2/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.14.1](/python/docs/reference/dialogflow/2.14.1/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.13.0](/python/docs/reference/dialogflow/2.13.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.12.0](/python/docs/reference/dialogflow/2.12.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.11.0](/python/docs/reference/dialogflow/2.11.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.10.0](/python/docs/reference/dialogflow/2.10.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.9.1](/python/docs/reference/dialogflow/2.9.1/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.8.1](/python/docs/reference/dialogflow/2.8.1/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.7.1](/python/docs/reference/dialogflow/2.7.1/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.6.0](/python/docs/reference/dialogflow/2.6.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.5.0](/python/docs/reference/dialogflow/2.5.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.4.0](/python/docs/reference/dialogflow/2.4.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.3.0](/python/docs/reference/dialogflow/2.3.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.2.0](/python/docs/reference/dialogflow/2.2.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.1.2](/python/docs/reference/dialogflow/2.1.2/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [2.0.0](/python/docs/reference/dialogflow/2.0.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [1.1.3](/python/docs/reference/dialogflow/1.1.3/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [1.0.0](/python/docs/reference/dialogflow/1.0.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [0.8.0](/python/docs/reference/dialogflow/0.8.0/google.cloud.dialogflow_v2.types.InferenceParameter)\n- [0.7.2](/python/docs/reference/dialogflow/0.7.2/google.cloud.dialogflow_v2.types.InferenceParameter) \n\n InferenceParameter(mapping=None, *, ignore_unknown_fields=False, **kwargs)\n\nThe parameters of inference.\n\n.. _oneof: \u003chttps://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields\u003e"]]