Google Cloud Ai Platform V1 Client - Class SpeculativeDecodingSpec (1.17.0)

Reference documentation and code samples for the Google Cloud Ai Platform V1 Client class SpeculativeDecodingSpec.

Configuration for Speculative Decoding.

Generated from protobuf message google.cloud.aiplatform.v1.SpeculativeDecodingSpec

Namespace

Google \ Cloud \ AIPlatform \ V1

Methods

__construct

Constructor.

Parameters
Name Description
data array

Optional. Data for populating the Message object.
↳ draft_model_speculation SpeculativeDecodingSpec\DraftModelSpeculation

draft model speculation.
↳ ngram_speculation SpeculativeDecodingSpec\NgramSpeculation

N-Gram speculation.
↳ speculative_token_count int

The number of speculative tokens to generate at each step.

getDraftModelSpeculation

draft model speculation.

Returns
Type Description
SpeculativeDecodingSpec\DraftModelSpeculation|null

hasDraftModelSpeculation

setDraftModelSpeculation

draft model speculation.

Parameter
Name Description
var SpeculativeDecodingSpec\DraftModelSpeculation
Returns
Type Description
$this

getNgramSpeculation

N-Gram speculation.

Returns
Type Description
SpeculativeDecodingSpec\NgramSpeculation|null

hasNgramSpeculation

setNgramSpeculation

N-Gram speculation.

Parameter
Name Description
var SpeculativeDecodingSpec\NgramSpeculation
Returns
Type Description
$this

getSpeculativeTokenCount

The number of speculative tokens to generate at each step.

Returns
Type Description
int

setSpeculativeTokenCount

The number of speculative tokens to generate at each step.

Parameter
Name Description
var int
Returns
Type Description
$this

getSpeculation

Returns
Type Description
string