public sealed class SpeculativeDecodingSpec.Types.NgramSpeculation : IMessage<SpeculativeDecodingSpec.Types.NgramSpeculation>, IEquatable<SpeculativeDecodingSpec.Types.NgramSpeculation>, IDeepCloneable<SpeculativeDecodingSpec.Types.NgramSpeculation>, IBufferMessage, IMessageReference documentation and code samples for the Vertex AI v1beta1 API class SpeculativeDecodingSpec.Types.NgramSpeculation.
N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.
Implements
IMessageSpeculativeDecodingSpecTypesNgramSpeculation, IEquatableSpeculativeDecodingSpecTypesNgramSpeculation, IDeepCloneableSpeculativeDecodingSpecTypesNgramSpeculation, IBufferMessage, IMessageNamespace
Google.Cloud.AIPlatform.V1Beta1Assembly
Google.Cloud.AIPlatform.V1Beta1.dll
Constructors
NgramSpeculation()
public NgramSpeculation()NgramSpeculation(NgramSpeculation)
public NgramSpeculation(SpeculativeDecodingSpec.Types.NgramSpeculation other)| Parameter | |
|---|---|
| Name | Description |
other |
SpeculativeDecodingSpecTypesNgramSpeculation |
Properties
NgramSize
public int NgramSize { get; set; }The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.
| Property Value | |
|---|---|
| Type | Description |
int |
|