- 1.100.0 (latest)
- 1.99.0
- 1.98.0
- 1.97.0
- 1.96.0
- 1.95.1
- 1.94.0
- 1.93.1
- 1.92.0
- 1.91.0
- 1.90.0
- 1.89.0
- 1.88.0
- 1.87.0
- 1.86.0
- 1.85.0
- 1.84.0
- 1.83.0
- 1.82.0
- 1.81.0
- 1.80.0
- 1.79.0
- 1.78.0
- 1.77.0
- 1.76.0
- 1.75.0
- 1.74.0
- 1.73.0
- 1.72.0
- 1.71.1
- 1.70.0
- 1.69.0
- 1.68.0
- 1.67.1
- 1.66.0
- 1.65.0
- 1.63.0
- 1.62.0
- 1.60.0
- 1.59.0
- 1.58.0
- 1.57.0
- 1.56.0
- 1.55.0
- 1.54.1
- 1.53.0
- 1.52.0
- 1.51.0
- 1.50.0
- 1.49.0
- 1.48.0
- 1.47.0
- 1.46.0
- 1.45.0
- 1.44.0
- 1.43.0
- 1.39.0
- 1.38.1
- 1.37.0
- 1.36.4
- 1.35.0
- 1.34.0
- 1.33.1
- 1.32.0
- 1.31.1
- 1.30.1
- 1.29.0
- 1.28.1
- 1.27.1
- 1.26.1
- 1.25.0
- 1.24.1
- 1.23.0
- 1.22.1
- 1.21.0
- 1.20.0
- 1.19.1
- 1.18.3
- 1.17.1
- 1.16.1
- 1.15.1
- 1.14.0
- 1.13.1
- 1.12.1
- 1.11.0
- 1.10.0
- 1.9.0
- 1.8.1
- 1.7.1
- 1.6.2
- 1.5.0
- 1.4.3
- 1.3.0
- 1.2.0
- 1.1.1
- 1.0.1
- 0.9.0
- 0.8.0
- 0.7.1
- 0.6.0
- 0.5.1
- 0.4.0
- 0.3.1
SpeculativeDecodingSpec(mapping=None, *, ignore_unknown_fields=False, **kwargs)Configuration for Speculative Decoding.
This message has oneof_ fields (mutually exclusive fields).
For each oneof, at most one member field can be set at the same time.
Setting any member of the oneof automatically clears all other
members.
.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields
Attributes |
|
|---|---|
| Name | Description |
draft_model_speculation |
google.cloud.aiplatform_v1.types.SpeculativeDecodingSpec.DraftModelSpeculation
draft model speculation. This field is a member of oneof_ speculation.
|
ngram_speculation |
google.cloud.aiplatform_v1.types.SpeculativeDecodingSpec.NgramSpeculation
N-Gram speculation. This field is a member of oneof_ speculation.
|
speculative_token_count |
int
The number of speculative tokens to generate at each step. |
Classes
DraftModelSpeculation
DraftModelSpeculation(mapping=None, *, ignore_unknown_fields=False, **kwargs)Draft model speculation works by using the smaller model to generate candidate tokens for speculative decoding.
NgramSpeculation
NgramSpeculation(mapping=None, *, ignore_unknown_fields=False, **kwargs)N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.
Methods
SpeculativeDecodingSpec
SpeculativeDecodingSpec(mapping=None, *, ignore_unknown_fields=False, **kwargs)Configuration for Speculative Decoding.
This message has oneof_ fields (mutually exclusive fields).
For each oneof, at most one member field can be set at the same time.
Setting any member of the oneof automatically clears all other
members.
.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields