Class Google::Cloud::Dataproc::V1::ClusterConfig (v0.8.0)

The cluster config.

Inherits

  • Object

Extended By

  • Google::Protobuf::MessageExts::ClassMethods

Includes

  • Google::Protobuf::MessageExts

Methods

#autoscaling_config

def autoscaling_config() -> ::Google::Cloud::Dataproc::V1::AutoscalingConfig
Returns

#autoscaling_config=

def autoscaling_config=(value) -> ::Google::Cloud::Dataproc::V1::AutoscalingConfig
Parameter
Returns

#config_bucket

def config_bucket() -> ::String
Returns
  • (::String) — Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

#config_bucket=

def config_bucket=(value) -> ::String
Parameter
  • value (::String) — Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
Returns
  • (::String) — Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

#encryption_config

def encryption_config() -> ::Google::Cloud::Dataproc::V1::EncryptionConfig
Returns

#encryption_config=

def encryption_config=(value) -> ::Google::Cloud::Dataproc::V1::EncryptionConfig
Parameter
Returns

#endpoint_config

def endpoint_config() -> ::Google::Cloud::Dataproc::V1::EndpointConfig
Returns

#endpoint_config=

def endpoint_config=(value) -> ::Google::Cloud::Dataproc::V1::EndpointConfig
Parameter
Returns

#gce_cluster_config

def gce_cluster_config() -> ::Google::Cloud::Dataproc::V1::GceClusterConfig
Returns

#gce_cluster_config=

def gce_cluster_config=(value) -> ::Google::Cloud::Dataproc::V1::GceClusterConfig
Parameter
Returns

#gke_cluster_config

def gke_cluster_config() -> ::Google::Cloud::Dataproc::V1::GkeClusterConfig
Returns
  • (::Google::Cloud::Dataproc::V1::GkeClusterConfig) — Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.

#gke_cluster_config=

def gke_cluster_config=(value) -> ::Google::Cloud::Dataproc::V1::GkeClusterConfig
Parameter
  • value (::Google::Cloud::Dataproc::V1::GkeClusterConfig) — Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
Returns
  • (::Google::Cloud::Dataproc::V1::GkeClusterConfig) — Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.

#initialization_actions

def initialization_actions() -> ::Array<::Google::Cloud::Dataproc::V1::NodeInitializationAction>
Returns
  • (::Array<::Google::Cloud::Dataproc::V1::NodeInitializationAction>) —

    Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget):

    ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
  ... master specific actions ...
else
  ... worker specific actions ...
fi

#initialization_actions=

def initialization_actions=(value) -> ::Array<::Google::Cloud::Dataproc::V1::NodeInitializationAction>
Parameter
  • value (::Array<::Google::Cloud::Dataproc::V1::NodeInitializationAction>) —

    Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget):

    ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
  ... master specific actions ...
else
  ... worker specific actions ...
fi
Returns
  • (::Array<::Google::Cloud::Dataproc::V1::NodeInitializationAction>) —

    Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget):

    ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
  ... master specific actions ...
else
  ... worker specific actions ...
fi

#lifecycle_config

def lifecycle_config() -> ::Google::Cloud::Dataproc::V1::LifecycleConfig
Returns

#lifecycle_config=

def lifecycle_config=(value) -> ::Google::Cloud::Dataproc::V1::LifecycleConfig
Parameter
Returns

#master_config

def master_config() -> ::Google::Cloud::Dataproc::V1::InstanceGroupConfig
Returns

#master_config=

def master_config=(value) -> ::Google::Cloud::Dataproc::V1::InstanceGroupConfig
Parameter
Returns

#metastore_config

def metastore_config() -> ::Google::Cloud::Dataproc::V1::MetastoreConfig
Returns

#metastore_config=

def metastore_config=(value) -> ::Google::Cloud::Dataproc::V1::MetastoreConfig
Parameter
Returns

#secondary_worker_config

def secondary_worker_config() -> ::Google::Cloud::Dataproc::V1::InstanceGroupConfig
Returns

#secondary_worker_config=

def secondary_worker_config=(value) -> ::Google::Cloud::Dataproc::V1::InstanceGroupConfig
Parameter
Returns

#security_config

def security_config() -> ::Google::Cloud::Dataproc::V1::SecurityConfig
Returns

#security_config=

def security_config=(value) -> ::Google::Cloud::Dataproc::V1::SecurityConfig
Parameter
Returns

#software_config

def software_config() -> ::Google::Cloud::Dataproc::V1::SoftwareConfig
Returns

#software_config=

def software_config=(value) -> ::Google::Cloud::Dataproc::V1::SoftwareConfig
Parameter
Returns

#temp_bucket

def temp_bucket() -> ::String
Returns
  • (::String) — Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

#temp_bucket=

def temp_bucket=(value) -> ::String
Parameter
  • value (::String) — Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
Returns
  • (::String) — Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

#worker_config

def worker_config() -> ::Google::Cloud::Dataproc::V1::InstanceGroupConfig
Returns

#worker_config=

def worker_config=(value) -> ::Google::Cloud::Dataproc::V1::InstanceGroupConfig
Parameter
Returns