Class WorkflowTemplateServiceClient (0.7.0)

WorkflowTemplateServiceClient(
    transport=None,
    channel=None,
    credentials=None,
    client_config=None,
    client_info=None,
    client_options=None,
)

The API interface for managing Workflow Templates in the Dataproc API.

Methods

WorkflowTemplateServiceClient

WorkflowTemplateServiceClient(
    transport=None,
    channel=None,
    credentials=None,
    client_config=None,
    client_info=None,
    client_options=None,
)

Constructor.

Parameters
NameDescription
channel grpc.Channel

DEPRECATED. A Channel instance through which to make calls. This argument is mutually exclusive with credentials; providing both will raise an exception.

credentials google.auth.credentials.Credentials

The authorization credentials to attach to requests. These credentials identify this application to the service. If none are specified, the client will attempt to ascertain the credentials from the environment. This argument is mutually exclusive with providing a transport instance to transport; doing so will raise an exception.

client_config dict

DEPRECATED. A dictionary of call options for each method. If not specified, the default configuration is used.

client_info google.api_core.gapic_v1.client_info.ClientInfo

The client info used to send a user-agent string along with API requests. If None, then default info will be used. Generally, you only need to set this if you're developing your own client library.

client_options Union[dict, google.api_core.client_options.ClientOptions]

Client options used to set user options on the client. API Endpoint should be set through client_options.

create_workflow_template

create_workflow_template(parent, template, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Creates new workflow template.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.WorkflowTemplateServiceClient()

parent = client.region_path('[PROJECT]', '[REGION]')

TODO: Initialize template:

template = {}

response = client.create_workflow_template(parent, template)

Parameters
NameDescription
parent str

Required. The resource name of the region or location, as described in https://cloud.google.com/apis/design/resource_names. - For projects.regions.workflowTemplates,create, the resource name of the region has the following format: projects/{project_id}/regions/{region} - For projects.locations.workflowTemplates.create, the resource name of the location has the following format: projects/{project_id}/locations/{location}

template Union[dict, WorkflowTemplate]

Required. The Dataproc workflow template to create. If a dict is provided, it must be of the same form as the protobuf message WorkflowTemplate

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

delete_workflow_template

delete_workflow_template(name, version=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Deletes a workflow template. It does not cancel in-progress workflows.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.WorkflowTemplateServiceClient()

name = client.workflow_template_path('[PROJECT]', '[REGION]', '[WORKFLOW_TEMPLATE]')

client.delete_workflow_template(name)

Parameters
NameDescription
name str

Required. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. - For projects.regions.workflowTemplates.delete, the resource name of the template has the following format: projects/{project_id}/regions/{region}/workflowTemplates/{template_id} - For projects.locations.workflowTemplates.instantiate, the resource name of the template has the following format: projects/{project_id}/locations/{location}/workflowTemplates/{template_id}

version int

Optional. The version of workflow template to delete. If specified, will only delete the template if the current server version matches specified version.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

from_service_account_file

from_service_account_file(filename, *args, **kwargs)

Creates an instance of this client using the provided credentials file.

Parameter
NameDescription
filename str

The path to the service account private key json file.

Returns
TypeDescription
WorkflowTemplateServiceClientThe constructed client.

from_service_account_json

from_service_account_json(filename, *args, **kwargs)

Creates an instance of this client using the provided credentials file.

Parameter
NameDescription
filename str

The path to the service account private key json file.

Returns
TypeDescription
WorkflowTemplateServiceClientThe constructed client.

get_workflow_template

get_workflow_template(name, version=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Retrieves the latest workflow template.

Can retrieve previously instantiated template by specifying optional version parameter.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.WorkflowTemplateServiceClient()

name = client.workflow_template_path('[PROJECT]', '[REGION]', '[WORKFLOW_TEMPLATE]')

response = client.get_workflow_template(name)

Parameters
NameDescription
name str

Required. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. - For projects.regions.workflowTemplates.get, the resource name of the template has the following format: projects/{project_id}/regions/{region}/workflowTemplates/{template_id} - For projects.locations.workflowTemplates.get, the resource name of the template has the following format: projects/{project_id}/locations/{location}/workflowTemplates/{template_id}

version int

Optional. The version of workflow template to retrieve. Only previously instantiated versions can be retrieved. If unspecified, retrieves the current version.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

instantiate_inline_workflow_template

instantiate_inline_workflow_template(parent, template, request_id=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Instantiates a template and begins execution.

This method is equivalent to executing the sequence CreateWorkflowTemplate, InstantiateWorkflowTemplate, DeleteWorkflowTemplate.

The returned Operation can be used to track execution of workflow by polling operations.get. The Operation will complete when entire workflow is finished.

The running workflow can be aborted via operations.cancel. This will cause any inflight jobs to be cancelled and workflow-owned clusters to be deleted.

The Operation.metadata will be WorkflowMetadata <https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1#workflowmetadata>. Also see Using WorkflowMetadata <https://cloud.google.com/dataproc/docs/concepts/workflows/debugging#using_workflowmetadata>.

On successful completion, Operation.response will be Empty.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.WorkflowTemplateServiceClient()

parent = client.region_path('[PROJECT]', '[REGION]')

TODO: Initialize template:

template = {}

response = client.instantiate_inline_workflow_template(parent, template)

def callback(operation_future): ... # Handle result. ... result = operation_future.result()

response.add_done_callback(callback)

Handle metadata.

metadata = response.metadata()

Parameters
NameDescription
parent str

Required. The resource name of the region or location, as described in https://cloud.google.com/apis/design/resource_names. - For projects.regions.workflowTemplates,instantiateinline, the resource name of the region has the following format: projects/{project_id}/regions/{region} - For projects.locations.workflowTemplates.instantiateinline, the resource name of the location has the following format: projects/{project_id}/locations/{location}

template Union[dict, WorkflowTemplate]

Required. The workflow template to instantiate. If a dict is provided, it must be of the same form as the protobuf message WorkflowTemplate

request_id str

Optional. A tag that prevents multiple concurrent workflow instances with the same tag from running. This mitigates risk of concurrent instances started due to retries. It is recommended to always set this value to a UUID https://en.wikipedia.org/wiki/Universally_unique_identifier_. The tag must contain only letters (a-z, A-Z), numbers (0-9), underscores (\), and hyphens (-). The maximum length is 40 characters.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

instantiate_workflow_template

instantiate_workflow_template(name, version=None, request_id=None, parameters=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Instantiates a template and begins execution.

The returned Operation can be used to track execution of workflow by polling operations.get. The Operation will complete when entire workflow is finished.

The running workflow can be aborted via operations.cancel. This will cause any inflight jobs to be cancelled and workflow-owned clusters to be deleted.

The Operation.metadata will be WorkflowMetadata <https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1#workflowmetadata>. Also see Using WorkflowMetadata <https://cloud.google.com/dataproc/docs/concepts/workflows/debugging#using_workflowmetadata>.

On successful completion, Operation.response will be Empty.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.WorkflowTemplateServiceClient()

name = client.workflow_template_path('[PROJECT]', '[REGION]', '[WORKFLOW_TEMPLATE]')

response = client.instantiate_workflow_template(name)

def callback(operation_future): ... # Handle result. ... result = operation_future.result()

response.add_done_callback(callback)

Handle metadata.

metadata = response.metadata()

Parameters
NameDescription
name str

Required. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. - For projects.regions.workflowTemplates.instantiate, the resource name of the template has the following format: projects/{project_id}/regions/{region}/workflowTemplates/{template_id} - For projects.locations.workflowTemplates.instantiate, the resource name of the template has the following format: projects/{project_id}/locations/{location}/workflowTemplates/{template_id}

version int

Optional. The version of workflow template to instantiate. If specified, the workflow will be instantiated only if the current version of the workflow template has the supplied version. This option cannot be used to instantiate a previous version of workflow template.

request_id str

Optional. A tag that prevents multiple concurrent workflow instances with the same tag from running. This mitigates risk of concurrent instances started due to retries. It is recommended to always set this value to a UUID https://en.wikipedia.org/wiki/Universally_unique_identifier_. The tag must contain only letters (a-z, A-Z), numbers (0-9), underscores (\), and hyphens (-). The maximum length is 40 characters.

parameters dict[str -> str]

Optional. Map from parameter names to values that should be used for those parameters. Values may not exceed 100 characters.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

list_workflow_templates

list_workflow_templates(parent, page_size=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Lists workflows that match the specified filter in the request.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.WorkflowTemplateServiceClient()

parent = client.region_path('[PROJECT]', '[REGION]')

Iterate over all results

for element in client.list_workflow_templates(parent): ... # process element ... pass

Alternatively:

Iterate over results one page at a time

for page in client.list_workflow_templates(parent).pages: ... for element in page: ... # process element ... pass

Parameters
NameDescription
parent str

Required. The resource name of the region or location, as described in https://cloud.google.com/apis/design/resource_names. - For projects.regions.workflowTemplates,list, the resource name of the region has the following format: projects/{project_id}/regions/{region} - For projects.locations.workflowTemplates.list, the resource name of the location has the following format: projects/{project_id}/locations/{location}

page_size int

The maximum number of resources contained in the underlying API response. If page streaming is performed per- resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

region_path

region_path(project, region)

Return a fully-qualified region string.

update_workflow_template

update_workflow_template(template, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Updates (replaces) workflow template. The updated template must contain version that matches the current server version.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.WorkflowTemplateServiceClient()

TODO: Initialize template:

template = {}

response = client.update_workflow_template(template)

Parameters
NameDescription
template Union[dict, WorkflowTemplate]

Required. The updated workflow template. The template.version field must match the current version. If a dict is provided, it must be of the same form as the protobuf message WorkflowTemplate

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

workflow_template_path

workflow_template_path(project, region, workflow_template)

Return a fully-qualified workflow_template string.