Cloud AI Platform v1beta1 API - Class PredictionService.PredictionServiceBase (1.0.0-beta12)

[BindServiceMethod(typeof(PredictionService), "BindService")]
public abstract class PredictionService.PredictionServiceBase

Reference documentation and code samples for the Cloud AI Platform v1beta1 API class PredictionService.PredictionServiceBase.

Base class for server-side implementations of PredictionService

Inheritance

object > PredictionService.PredictionServiceBase

Inherited Members

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ToString()

Namespace

Google.Cloud.AIPlatform.V1Beta1

Assembly

Google.Cloud.AIPlatform.V1Beta1.dll

Methods

ChatCompletions(ChatCompletionsRequest, IServerStreamWriter<HttpBody>, ServerCallContext)

public virtual Task ChatCompletions(ChatCompletionsRequest request, IServerStreamWriter<HttpBody> responseStream, ServerCallContext context)

Exposes an OpenAI-compatible endpoint for chat completions.

Parameters
Name	Description
`request`	`ChatCompletionsRequest` The request received from the client.
`responseStream`	`IServerStreamWriterHttpBody` Used for sending responses back to the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`Task`	A task indicating completion of the handler.

CountTokens(CountTokensRequest, ServerCallContext)

public virtual Task<CountTokensResponse> CountTokens(CountTokensRequest request, ServerCallContext context)

Perform a token counting.

Parameters
Name	Description
`request`	`CountTokensRequest` The request received from the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`TaskCountTokensResponse`	The response to send back to the client (wrapped by a task).

DirectPredict(DirectPredictRequest, ServerCallContext)

public virtual Task<DirectPredictResponse> DirectPredict(DirectPredictRequest request, ServerCallContext context)

Perform an unary online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameters
Name	Description
`request`	`DirectPredictRequest` The request received from the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`TaskDirectPredictResponse`	The response to send back to the client (wrapped by a task).

DirectRawPredict(DirectRawPredictRequest, ServerCallContext)

public virtual Task<DirectRawPredictResponse> DirectRawPredict(DirectRawPredictRequest request, ServerCallContext context)

Perform an unary online prediction request to a gRPC model server for custom containers.

Parameters
Name	Description
`request`	`DirectRawPredictRequest` The request received from the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`TaskDirectRawPredictResponse`	The response to send back to the client (wrapped by a task).

Explain(ExplainRequest, ServerCallContext)

public virtual Task<ExplainResponse> Explain(ExplainRequest request, ServerCallContext context)

Perform an online explanation.

If [deployed_model_id][google.cloud.aiplatform.v1beta1.ExplainRequest.deployed_model_id] is specified, the corresponding DeployModel must have [explanation_spec][google.cloud.aiplatform.v1beta1.DeployedModel.explanation_spec] populated. If [deployed_model_id][google.cloud.aiplatform.v1beta1.ExplainRequest.deployed_model_id] is not specified, all DeployedModels must have [explanation_spec][google.cloud.aiplatform.v1beta1.DeployedModel.explanation_spec] populated.

Parameters
Name	Description
`request`	`ExplainRequest` The request received from the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`TaskExplainResponse`	The response to send back to the client (wrapped by a task).

GenerateContent(GenerateContentRequest, ServerCallContext)

public virtual Task<GenerateContentResponse> GenerateContent(GenerateContentRequest request, ServerCallContext context)

Generate content with multimodal inputs.

Parameters
Name	Description
`request`	`GenerateContentRequest` The request received from the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`TaskGenerateContentResponse`	The response to send back to the client (wrapped by a task).

Predict(PredictRequest, ServerCallContext)

public virtual Task<PredictResponse> Predict(PredictRequest request, ServerCallContext context)

Perform an online prediction.

Parameters
Name	Description
`request`	`PredictRequest` The request received from the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`TaskPredictResponse`	The response to send back to the client (wrapped by a task).

RawPredict(RawPredictRequest, ServerCallContext)

public virtual Task<HttpBody> RawPredict(RawPredictRequest request, ServerCallContext context)

Perform an online prediction with an arbitrary HTTP payload.

The response includes the following HTTP headers:

X-Vertex-AI-Endpoint-Id: ID of the [Endpoint][google.cloud.aiplatform.v1beta1.Endpoint] that served this prediction.
X-Vertex-AI-Deployed-Model-Id: ID of the Endpoint's [DeployedModel][google.cloud.aiplatform.v1beta1.DeployedModel] that served this prediction.

Parameters
Name	Description
`request`	`RawPredictRequest` The request received from the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`TaskHttpBody`	The response to send back to the client (wrapped by a task).

ServerStreamingPredict(StreamingPredictRequest, IServerStreamWriter<StreamingPredictResponse>, ServerCallContext)

public virtual Task ServerStreamingPredict(StreamingPredictRequest request, IServerStreamWriter<StreamingPredictResponse> responseStream, ServerCallContext context)

Perform a server-side streaming online prediction request for Vertex LLM streaming.

Parameters
Name	Description
`request`	`StreamingPredictRequest` The request received from the client.
`responseStream`	`IServerStreamWriterStreamingPredictResponse` Used for sending responses back to the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`Task`	A task indicating completion of the handler.

StreamDirectPredict(IAsyncStreamReader<StreamDirectPredictRequest>, IServerStreamWriter<StreamDirectPredictResponse>, ServerCallContext)

public virtual Task StreamDirectPredict(IAsyncStreamReader<StreamDirectPredictRequest> requestStream, IServerStreamWriter<StreamDirectPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameters
Name	Description
`requestStream`	`IAsyncStreamReaderStreamDirectPredictRequest` Used for reading requests from the client.
`responseStream`	`IServerStreamWriterStreamDirectPredictResponse` Used for sending responses back to the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`Task`	A task indicating completion of the handler.

StreamDirectRawPredict(IAsyncStreamReader<StreamDirectRawPredictRequest>, IServerStreamWriter<StreamDirectRawPredictResponse>, ServerCallContext)

public virtual Task StreamDirectRawPredict(IAsyncStreamReader<StreamDirectRawPredictRequest> requestStream, IServerStreamWriter<StreamDirectRawPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request to a gRPC model server for custom containers.

Parameters
Name	Description
`requestStream`	`IAsyncStreamReaderStreamDirectRawPredictRequest` Used for reading requests from the client.
`responseStream`	`IServerStreamWriterStreamDirectRawPredictResponse` Used for sending responses back to the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`Task`	A task indicating completion of the handler.

StreamGenerateContent(GenerateContentRequest, IServerStreamWriter<GenerateContentResponse>, ServerCallContext)

public virtual Task StreamGenerateContent(GenerateContentRequest request, IServerStreamWriter<GenerateContentResponse> responseStream, ServerCallContext context)

Generate content with multimodal inputs with streaming support.

Parameters
Name	Description
`request`	`GenerateContentRequest` The request received from the client.
`responseStream`	`IServerStreamWriterGenerateContentResponse` Used for sending responses back to the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`Task`	A task indicating completion of the handler.

StreamRawPredict(StreamRawPredictRequest, IServerStreamWriter<HttpBody>, ServerCallContext)

public virtual Task StreamRawPredict(StreamRawPredictRequest request, IServerStreamWriter<HttpBody> responseStream, ServerCallContext context)

Perform a streaming online prediction with an arbitrary HTTP payload.

Parameters
Name	Description
`request`	`StreamRawPredictRequest` The request received from the client.
`responseStream`	`IServerStreamWriterHttpBody` Used for sending responses back to the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`Task`	A task indicating completion of the handler.

StreamingPredict(IAsyncStreamReader<StreamingPredictRequest>, IServerStreamWriter<StreamingPredictResponse>, ServerCallContext)

public virtual Task StreamingPredict(IAsyncStreamReader<StreamingPredictRequest> requestStream, IServerStreamWriter<StreamingPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request for Vertex first-party products and frameworks.

Parameters
Name	Description
`requestStream`	`IAsyncStreamReaderStreamingPredictRequest` Used for reading requests from the client.
`responseStream`	`IServerStreamWriterStreamingPredictResponse` Used for sending responses back to the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`Task`	A task indicating completion of the handler.

StreamingRawPredict(IAsyncStreamReader<StreamingRawPredictRequest>, IServerStreamWriter<StreamingRawPredictResponse>, ServerCallContext)

public virtual Task StreamingRawPredict(IAsyncStreamReader<StreamingRawPredictRequest> requestStream, IServerStreamWriter<StreamingRawPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request through gRPC.

Parameters
Name	Description
`requestStream`	`IAsyncStreamReaderStreamingRawPredictRequest` Used for reading requests from the client.
`responseStream`	`IServerStreamWriterStreamingRawPredictResponse` Used for sending responses back to the client.
`context`	`ServerCallContext` The context of the server-side call handler being invoked.

Returns
Type	Description
`Task`	A task indicating completion of the handler.