Dataflow V1beta3 API - Class Google::Cloud::Dataflow::V1beta3::Step (v0.13.1)

Reference documentation and code samples for the Dataflow V1beta3 API class Google::Cloud::Dataflow::V1beta3::Step.

Defines a particular step within a Cloud Dataflow job.

A job consists of multiple steps, each of which performs some specific operation as part of the overall job. Data is typically passed from one step to another as part of the job.

Note: The properties of this object are not stable and might change.

Here's an example of a sequence of steps which together implement a Map-Reduce job:

  • Read a collection of data from some source, parsing the collection's elements.

  • Validate the elements.

  • Apply a user-defined function to map each element to some value and extract an element-specific key value.

  • Group elements with the same key into a single element with that key, transforming a multiply-keyed collection into a uniquely-keyed collection.

  • Write the elements out to some data sink.

Note that the Cloud Dataflow service may be used to run many different types of jobs, not just Map-Reduce.

Inherits

  • Object

Extended By

  • Google::Protobuf::MessageExts::ClassMethods

Includes

  • Google::Protobuf::MessageExts

Methods

#kind

def kind() -> ::String
Returns
  • (::String) — The kind of step in the Cloud Dataflow job.

#kind=

def kind=(value) -> ::String
Parameter
  • value (::String) — The kind of step in the Cloud Dataflow job.
Returns
  • (::String) — The kind of step in the Cloud Dataflow job.

#name

def name() -> ::String
Returns
  • (::String) — The name that identifies the step. This must be unique for each step with respect to all other steps in the Cloud Dataflow job.

#name=

def name=(value) -> ::String
Parameter
  • value (::String) — The name that identifies the step. This must be unique for each step with respect to all other steps in the Cloud Dataflow job.
Returns
  • (::String) — The name that identifies the step. This must be unique for each step with respect to all other steps in the Cloud Dataflow job.

#properties

def properties() -> ::Google::Protobuf::Struct
Returns
  • (::Google::Protobuf::Struct) — Named properties associated with the step. Each kind of predefined step has its own required set of properties. Must be provided on Create. Only retrieved with JOB_VIEW_ALL.

#properties=

def properties=(value) -> ::Google::Protobuf::Struct
Parameter
  • value (::Google::Protobuf::Struct) — Named properties associated with the step. Each kind of predefined step has its own required set of properties. Must be provided on Create. Only retrieved with JOB_VIEW_ALL.
Returns
  • (::Google::Protobuf::Struct) — Named properties associated with the step. Each kind of predefined step has its own required set of properties. Must be provided on Create. Only retrieved with JOB_VIEW_ALL.