Class BatchingSettings (2.33.0)

Represents the batching settings to use for an API method that is capable of batching.

By default the settings are configured to not use batching (i.e. the batch size threshold is 1). This is the safest default behavior, which has meaning in all possible scenarios. Users are expected to configure actual batching thresholds explicitly: the element count, the request bytes count and the delay.

Warning: With the wrong settings, it is possible to cause long periods of dead waiting time.

When batching is turned on for an API method, a call to that method will result in the request being queued up with other requests. When any of the set thresholds are reached, the queued up requests are packaged together in a batch and set to the service as a single RPC. When the response comes back, it is split apart into individual responses according to the individual input requests.

There are several supported thresholds:

• Delay Threshold: Counting from the time that the first message is queued, once this delay has passed, then send the batch. The default value is 1 millisecond.
• Message Count Threshold: Once this many messages are queued, send all of the messages in a single call, even if the delay threshold hasn't elapsed yet. The default value is 1 message.
• Request Byte Threshold: Once the number of bytes in the batched request reaches this threshold, send all of the messages in a single call, even if neither the delay or message count thresholds have been exceeded yet. The default value is 1 byte.

These thresholds are treated as triggers, not as limits. Thus, if a request is made with 2x the message count threshold, it will not be split apart (unless one of the limits listed further down is crossed); only one batch will be sent. Each threshold is an independent trigger and doesn't have any knowledge of the other thresholds.

Two of the values above also have limits:

• Message Count Limit: The limit of the number of messages that the server will accept in a single request.
• Request Byte Limit: The limit of the byte size of a request that the server will accept.

For these values, individual requests that surpass the limit are rejected, and the batching logic will not batch together requests if the resulting batch will surpass the limit. Thus, a batch can be sent that is actually under the threshold if the next request would put the combined request over the limit.

Batching also supports FlowControl. This can be used to prevent the batching implementation from accumulating messages without limit, resulting eventually in an OutOfMemory exception. This can occur if messages are created and added to batching faster than they can be processed. The flow control behavior is controlled using FlowControlSettings.

Get a new builder.

Get the delay threshold to use for batching.

Get the element count threshold to use for batching.

Get the flow control settings to use.

Returns the Boolean object to indicate if the batching is enabled. Default to true

Get the request byte threshold to use for batching.

Get a builder with the same values as this object.