Class InstanceGroupConfig (5.10.2)

InstanceGroupConfig(mapping=None, *, ignore_unknown_fields=False, **kwargs)

The config settings for Compute Engine resources in an instance group, such as a master or worker group.

Attributes

Name Description
num_instances int
Optional. The number of VM instances in the instance group. For `HA cluster `__ master_config <#fields.master_config>__ groups, **must be set to 3**. For standard cluster master_config <#fields.master_config>__ groups, **must be set to 1**.
instance_names MutableSequence[str]
Output only. The list of instance names. Dataproc derives the names from cluster_name, num_instances, and the instance group.
instance_references MutableSequence[google.cloud.dataproc_v1.types.InstanceReference]
Output only. List of references to Compute Engine instances.
image_uri str
Optional. The Compute Engine image resource used for cluster instances. The URI can represent an image or image family. Image examples: - https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/[image-id] - projects/[project_id]/global/images/[image-id] - image-id Image family examples. Dataproc will use the most recent image from the family: - https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/family/[custom-image-family-name] - projects/[project_id]/global/images/family/[custom-image-family-name] If the URI is unspecified, it will be inferred from SoftwareConfig.image_version or the system default.
machine_type_uri str
Optional. The Compute Engine machine type used for cluster instances. A full URL, partial URI, or short name are valid. Examples: - https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2 - projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2 - n1-standard-2 **Auto Zone Exception**: If you are using the Dataproc `Auto Zone Placement
disk_config google.cloud.dataproc_v1.types.DiskConfig
Optional. Disk option config settings.
is_preemptible bool
Output only. Specifies that this instance group contains preemptible instances.
preemptibility google.cloud.dataproc_v1.types.InstanceGroupConfig.Preemptibility
Optional. Specifies the preemptibility of the instance group. The default value for master and worker groups is NON_PREEMPTIBLE. This default cannot be changed. The default value for secondary instances is PREEMPTIBLE.
managed_group_config google.cloud.dataproc_v1.types.ManagedGroupConfig
Output only. The config for Compute Engine Instance Group Manager that manages this group. This is only used for preemptible instance groups.
accelerators MutableSequence[google.cloud.dataproc_v1.types.AcceleratorConfig]
Optional. The Compute Engine accelerator configuration for these instances.
min_cpu_platform str
Optional. Specifies the minimum cpu platform for the Instance Group. See `Dataproc -> Minimum CPU Platform
min_num_instances int
Optional. The minimum number of primary worker instances to create. If min_num_instances is set, cluster creation will succeed if the number of primary workers created is at least equal to the min_num_instances number. Example: Cluster creation request with num_instances = 5 and min_num_instances = 3: - If 4 VMs are created and 1 instance fails, the failed VM is deleted. The cluster is resized to 4 instances and placed in a RUNNING state. - If 2 instances are created and 3 instances fail, the cluster in placed in an ERROR state. The failed VMs are not deleted.
instance_flexibility_policy google.cloud.dataproc_v1.types.InstanceFlexibilityPolicy
Optional. Instance flexibility Policy allowing a mixture of VM shapes and provisioning models.
startup_config google.cloud.dataproc_v1.types.StartupConfig
Optional. Configuration to handle the startup of instances during cluster create and update process.

Classes

Preemptibility

Preemptibility(value)

Controls the use of preemptible instances within the group.

    This option is allowed for all instance groups
    and is the only valid value for Master and
    Worker instance groups.
PREEMPTIBLE (2):
    Instances are [preemptible]
    (https://cloud.google.com/compute/docs/instances/preemptible).

    This option is allowed only for [secondary worker]
    (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
    groups.
SPOT (3):
    Instances are [Spot VMs]
    (https://cloud.google.com/compute/docs/instances/spot).

    This option is allowed only for [secondary worker]
    (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
    groups. Spot VMs are the latest version of [preemptible VMs]
    (https://cloud.google.com/compute/docs/instances/preemptible),
    and provide additional features.