Dataflow V1beta3 API - Class Google::Cloud::Dataflow::V1beta3::WorkerPool (v0.9.1)

Reference documentation and code samples for the Dataflow V1beta3 API class Google::Cloud::Dataflow::V1beta3::WorkerPool.

Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.

Inherits

Object

Extended By

Google::Protobuf::MessageExts::ClassMethods

Includes

Google::Protobuf::MessageExts

Methods

#autoscaling_settings

def autoscaling_settings() -> ::Google::Cloud::Dataflow::V1beta3::AutoscalingSettings

Returns

(::Google::Cloud::Dataflow::V1beta3::AutoscalingSettings) — Settings for autoscaling of this WorkerPool.

#autoscaling_settings=

def autoscaling_settings=(value) -> ::Google::Cloud::Dataflow::V1beta3::AutoscalingSettings

Parameter

value (::Google::Cloud::Dataflow::V1beta3::AutoscalingSettings) — Settings for autoscaling of this WorkerPool.

Returns

(::Google::Cloud::Dataflow::V1beta3::AutoscalingSettings) — Settings for autoscaling of this WorkerPool.

#data_disks

def data_disks() -> ::Array<::Google::Cloud::Dataflow::V1beta3::Disk>

Returns

(::Array<::Google::Cloud::Dataflow::V1beta3::Disk>) — Data disks that are used by a VM in this workflow.

#data_disks=

def data_disks=(value) -> ::Array<::Google::Cloud::Dataflow::V1beta3::Disk>

Parameter

value (::Array<::Google::Cloud::Dataflow::V1beta3::Disk>) — Data disks that are used by a VM in this workflow.

Returns

(::Array<::Google::Cloud::Dataflow::V1beta3::Disk>) — Data disks that are used by a VM in this workflow.

#default_package_set

def default_package_set() -> ::Google::Cloud::Dataflow::V1beta3::DefaultPackageSet

Returns

(::Google::Cloud::Dataflow::V1beta3::DefaultPackageSet) — The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.

#default_package_set=

def default_package_set=(value) -> ::Google::Cloud::Dataflow::V1beta3::DefaultPackageSet

Parameter

value (::Google::Cloud::Dataflow::V1beta3::DefaultPackageSet) — The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.

Returns

(::Google::Cloud::Dataflow::V1beta3::DefaultPackageSet) — The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.

#disk_size_gb

def disk_size_gb() -> ::Integer

Returns

(::Integer) — Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

#disk_size_gb=

def disk_size_gb=(value) -> ::Integer

Parameter

value (::Integer) — Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

Returns

(::Integer) — Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

#disk_source_image

def disk_source_image() -> ::String

Returns

(::String) — Fully qualified source image for disks.

#disk_source_image=

def disk_source_image=(value) -> ::String

Parameter

value (::String) — Fully qualified source image for disks.

Returns

(::String) — Fully qualified source image for disks.

#disk_type

def disk_type() -> ::String

Returns

(::String) — Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

#disk_type=

def disk_type=(value) -> ::String

Parameter

value (::String) — Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

Returns

(::String) — Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

#ip_configuration

def ip_configuration() -> ::Google::Cloud::Dataflow::V1beta3::WorkerIPAddressConfiguration

Returns

(::Google::Cloud::Dataflow::V1beta3::WorkerIPAddressConfiguration) — Configuration for VM IPs.

#ip_configuration=

def ip_configuration=(value) -> ::Google::Cloud::Dataflow::V1beta3::WorkerIPAddressConfiguration

Parameter

value (::Google::Cloud::Dataflow::V1beta3::WorkerIPAddressConfiguration) — Configuration for VM IPs.

Returns

(::Google::Cloud::Dataflow::V1beta3::WorkerIPAddressConfiguration) — Configuration for VM IPs.

#kind

def kind() -> ::String

Returns

(::String) — The kind of the worker pool; currently only harness and shuffle are supported.

#kind=

def kind=(value) -> ::String

Parameter

value (::String) — The kind of the worker pool; currently only harness and shuffle are supported.

Returns

(::String) — The kind of the worker pool; currently only harness and shuffle are supported.

#machine_type

def machine_type() -> ::String

Returns

(::String) — Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

#machine_type=

def machine_type=(value) -> ::String

Parameter

value (::String) — Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

Returns

(::String) — Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

#metadata

def metadata() -> ::Google::Protobuf::Map{::String => ::String}

Returns

(::Google::Protobuf::Map{::String => ::String}) — Metadata to set on the Google Compute Engine VMs.

#metadata=

def metadata=(value) -> ::Google::Protobuf::Map{::String => ::String}

Parameter

value (::Google::Protobuf::Map{::String => ::String}) — Metadata to set on the Google Compute Engine VMs.

Returns

(::Google::Protobuf::Map{::String => ::String}) — Metadata to set on the Google Compute Engine VMs.

#network

def network() -> ::String

Returns

(::String) — Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

#network=

def network=(value) -> ::String

Parameter

value (::String) — Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

Returns

(::String) — Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

#num_threads_per_worker

def num_threads_per_worker() -> ::Integer

Returns

(::Integer) — The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

#num_threads_per_worker=

def num_threads_per_worker=(value) -> ::Integer

Parameter

value (::Integer) — The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

Returns

(::Integer) — The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

#num_workers

def num_workers() -> ::Integer

Returns

(::Integer) — Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

#num_workers=

def num_workers=(value) -> ::Integer

Parameter

value (::Integer) — Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

Returns

(::Integer) — Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

#on_host_maintenance

def on_host_maintenance() -> ::String

Returns

(::String) — The action to take on host maintenance, as defined by the Google Compute Engine API.

#on_host_maintenance=

def on_host_maintenance=(value) -> ::String

Parameter

value (::String) — The action to take on host maintenance, as defined by the Google Compute Engine API.

Returns

(::String) — The action to take on host maintenance, as defined by the Google Compute Engine API.

#packages

def packages() -> ::Array<::Google::Cloud::Dataflow::V1beta3::Package>

Returns

(::Array<::Google::Cloud::Dataflow::V1beta3::Package>) — Packages to be installed on workers.

#packages=

def packages=(value) -> ::Array<::Google::Cloud::Dataflow::V1beta3::Package>

Parameter

value (::Array<::Google::Cloud::Dataflow::V1beta3::Package>) — Packages to be installed on workers.

Returns

(::Array<::Google::Cloud::Dataflow::V1beta3::Package>) — Packages to be installed on workers.

#pool_args

def pool_args() -> ::Google::Protobuf::Any

Returns

(::Google::Protobuf::Any) — Extra arguments for this worker pool.

#pool_args=

def pool_args=(value) -> ::Google::Protobuf::Any

Parameter

value (::Google::Protobuf::Any) — Extra arguments for this worker pool.

Returns

(::Google::Protobuf::Any) — Extra arguments for this worker pool.

#sdk_harness_container_images

def sdk_harness_container_images() -> ::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>

Returns

(::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>) — Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

#sdk_harness_container_images=

def sdk_harness_container_images=(value) -> ::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>

Parameter

value (::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>) — Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

Returns

(::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>) — Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

#subnetwork

def subnetwork() -> ::String

Returns

(::String) — Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

#subnetwork=

def subnetwork=(value) -> ::String

Parameter

value (::String) — Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

Returns

(::String) — Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

#taskrunner_settings

def taskrunner_settings() -> ::Google::Cloud::Dataflow::V1beta3::TaskRunnerSettings

Returns

(::Google::Cloud::Dataflow::V1beta3::TaskRunnerSettings) — Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.

#taskrunner_settings=

def taskrunner_settings=(value) -> ::Google::Cloud::Dataflow::V1beta3::TaskRunnerSettings

Parameter

value (::Google::Cloud::Dataflow::V1beta3::TaskRunnerSettings) — Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.

Returns

(::Google::Cloud::Dataflow::V1beta3::TaskRunnerSettings) — Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.

#teardown_policy

def teardown_policy() -> ::Google::Cloud::Dataflow::V1beta3::TeardownPolicy

Returns

(::Google::Cloud::Dataflow::V1beta3::TeardownPolicy) — Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down.

If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs.

If unknown or unspecified, the service will attempt to choose a reasonable default.

#teardown_policy=

def teardown_policy=(value) -> ::Google::Cloud::Dataflow::V1beta3::TeardownPolicy

Parameter

value (::Google::Cloud::Dataflow::V1beta3::TeardownPolicy) — Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down.

If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs.

If unknown or unspecified, the service will attempt to choose a reasonable default.

Returns

(::Google::Cloud::Dataflow::V1beta3::TeardownPolicy) — Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down.

If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs.

If unknown or unspecified, the service will attempt to choose a reasonable default.

#worker_harness_container_image

def worker_harness_container_image() -> ::String

Returns

(::String) — Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

Deprecated for the Fn API path. Use sdk_harness_container_images instead.

#worker_harness_container_image=

def worker_harness_container_image=(value) -> ::String

Parameter

value (::String) — Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

Deprecated for the Fn API path. Use sdk_harness_container_images instead.

Returns

(::String) — Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

Deprecated for the Fn API path. Use sdk_harness_container_images instead.

#zone

def zone() -> ::String

Returns

(::String) — Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.

#zone=

def zone=(value) -> ::String

Parameter

value (::String) — Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.

Returns

(::String) — Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.