Cloud Dataproc V1beta2 API - Class Google::Cloud::Dataproc::V1beta2::ClusterConfig (v0.6.3)

Reference documentation and code samples for the Cloud Dataproc V1beta2 API class Google::Cloud::Dataproc::V1beta2::ClusterConfig.

The cluster config.

Inherits

Object

Extended By

Google::Protobuf::MessageExts::ClassMethods

Includes

Google::Protobuf::MessageExts

Methods

#autoscaling_config

def autoscaling_config() -> ::Google::Cloud::Dataproc::V1beta2::AutoscalingConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::AutoscalingConfig) — Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.

#autoscaling_config=

def autoscaling_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::AutoscalingConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::AutoscalingConfig) — Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.

Returns

(::Google::Cloud::Dataproc::V1beta2::AutoscalingConfig) — Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.

#config_bucket

def config_bucket() -> ::String

Returns

(::String) — Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket).

#config_bucket=

def config_bucket=(value) -> ::String

Parameter

value (::String) — Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket).

Returns

(::String) — Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket).

#encryption_config

def encryption_config() -> ::Google::Cloud::Dataproc::V1beta2::EncryptionConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::EncryptionConfig) — Optional. Encryption settings for the cluster.

#encryption_config=

def encryption_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::EncryptionConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::EncryptionConfig) — Optional. Encryption settings for the cluster.

Returns

(::Google::Cloud::Dataproc::V1beta2::EncryptionConfig) — Optional. Encryption settings for the cluster.

#endpoint_config

def endpoint_config() -> ::Google::Cloud::Dataproc::V1beta2::EndpointConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::EndpointConfig) — Optional. Port/endpoint configuration for this cluster

#endpoint_config=

def endpoint_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::EndpointConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::EndpointConfig) — Optional. Port/endpoint configuration for this cluster

Returns

(::Google::Cloud::Dataproc::V1beta2::EndpointConfig) — Optional. Port/endpoint configuration for this cluster

#gce_cluster_config

def gce_cluster_config() -> ::Google::Cloud::Dataproc::V1beta2::GceClusterConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::GceClusterConfig) — Optional. The shared Compute Engine config settings for all instances in a cluster.

#gce_cluster_config=

def gce_cluster_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::GceClusterConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::GceClusterConfig) — Optional. The shared Compute Engine config settings for all instances in a cluster.

Returns

(::Google::Cloud::Dataproc::V1beta2::GceClusterConfig) — Optional. The shared Compute Engine config settings for all instances in a cluster.

#gke_cluster_config

def gke_cluster_config() -> ::Google::Cloud::Dataproc::V1beta2::GkeClusterConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::GkeClusterConfig) — Optional. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.

#gke_cluster_config=

def gke_cluster_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::GkeClusterConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::GkeClusterConfig) — Optional. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.

Returns

(::Google::Cloud::Dataproc::V1beta2::GkeClusterConfig) — Optional. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.

#initialization_actions

def initialization_actions() -> ::Array<::Google::Cloud::Dataproc::V1beta2::NodeInitializationAction>

Returns

(::Array<::Google::Cloud::Dataproc::V1beta2::NodeInitializationAction>) —
Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget):
```
ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1beta2/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
  ... master specific actions ...
else
  ... worker specific actions ...
fi
```

#initialization_actions=

def initialization_actions=(value) -> ::Array<::Google::Cloud::Dataproc::V1beta2::NodeInitializationAction>

Parameter

value (::Array<::Google::Cloud::Dataproc::V1beta2::NodeInitializationAction>) —
Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget):
```
ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1beta2/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
  ... master specific actions ...
else
  ... worker specific actions ...
fi
```

Returns

(::Array<::Google::Cloud::Dataproc::V1beta2::NodeInitializationAction>) —
Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget):
```
ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1beta2/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
  ... master specific actions ...
else
  ... worker specific actions ...
fi
```

#lifecycle_config

def lifecycle_config() -> ::Google::Cloud::Dataproc::V1beta2::LifecycleConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::LifecycleConfig) — Optional. The config setting for auto delete cluster schedule.

#lifecycle_config=

def lifecycle_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::LifecycleConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::LifecycleConfig) — Optional. The config setting for auto delete cluster schedule.

Returns

(::Google::Cloud::Dataproc::V1beta2::LifecycleConfig) — Optional. The config setting for auto delete cluster schedule.

#master_config

def master_config() -> ::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig) — Optional. The Compute Engine config settings for the master instance in a cluster.

#master_config=

def master_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig) — Optional. The Compute Engine config settings for the master instance in a cluster.

Returns

(::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig) — Optional. The Compute Engine config settings for the master instance in a cluster.

#secondary_worker_config

def secondary_worker_config() -> ::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig) — Optional. The Compute Engine config settings for additional worker instances in a cluster.

#secondary_worker_config=

def secondary_worker_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig) — Optional. The Compute Engine config settings for additional worker instances in a cluster.

Returns

(::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig) — Optional. The Compute Engine config settings for additional worker instances in a cluster.

#security_config

def security_config() -> ::Google::Cloud::Dataproc::V1beta2::SecurityConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::SecurityConfig) — Optional. Security related configuration.

#security_config=

def security_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::SecurityConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::SecurityConfig) — Optional. Security related configuration.

Returns

(::Google::Cloud::Dataproc::V1beta2::SecurityConfig) — Optional. Security related configuration.

#software_config

def software_config() -> ::Google::Cloud::Dataproc::V1beta2::SoftwareConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::SoftwareConfig) — Optional. The config settings for software inside the cluster.

#software_config=

def software_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::SoftwareConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::SoftwareConfig) — Optional. The config settings for software inside the cluster.

Returns

(::Google::Cloud::Dataproc::V1beta2::SoftwareConfig) — Optional. The config settings for software inside the cluster.

#temp_bucket

def temp_bucket() -> ::String

Returns

(::String) — Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket.

#temp_bucket=

def temp_bucket=(value) -> ::String

Parameter

value (::String) — Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket.

Returns

(::String) — Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket.

#worker_config

def worker_config() -> ::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig

Returns

(::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig) — Optional. The Compute Engine config settings for worker instances in a cluster.

#worker_config=

def worker_config=(value) -> ::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig

Parameter

value (::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig) — Optional. The Compute Engine config settings for worker instances in a cluster.

Returns

(::Google::Cloud::Dataproc::V1beta2::InstanceGroupConfig) — Optional. The Compute Engine config settings for worker instances in a cluster.