google-cloud-dataflow overview (0.5.0)

com.google.dataflow.v1beta3

A client to Dataflow API

The interfaces provided are listed below, along with usage samples.

JobsV1Beta3Client

Service Description: Provides a method to create and modify Google Cloud Dataflow jobs. A Job is a multi-stage computation graph run by the Cloud Dataflow service.

Sample for JobsV1Beta3Client:


 try (JobsV1Beta3Client jobsV1Beta3Client = JobsV1Beta3Client.create()) {
   CreateJobRequest request =
       CreateJobRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setJob(Job.newBuilder().build())
           .setView(JobView.forNumber(0))
           .setReplaceJobId("replaceJobId441554788")
           .setLocation("location1901043637")
           .build();
   Job response = jobsV1Beta3Client.createJob(request);
 }

MessagesV1Beta3Client

Service Description: The Dataflow Messages API is used for monitoring the progress of Dataflow jobs.

Sample for MessagesV1Beta3Client:


 try (MessagesV1Beta3Client messagesV1Beta3Client = MessagesV1Beta3Client.create()) {
   ListJobMessagesRequest request =
       ListJobMessagesRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setJobId("jobId101296568")
           .setMinimumImportance(JobMessageImportance.forNumber(0))
           .setPageSize(883849137)
           .setPageToken("pageToken873572522")
           .setStartTime(Timestamp.newBuilder().build())
           .setEndTime(Timestamp.newBuilder().build())
           .setLocation("location1901043637")
           .build();
   for (JobMessage element : messagesV1Beta3Client.listJobMessages(request).iterateAll()) {
     // doThingsWith(element);
   }
 }

MetricsV1Beta3Client

Service Description: The Dataflow Metrics API lets you monitor the progress of Dataflow jobs.

Sample for MetricsV1Beta3Client:


 try (MetricsV1Beta3Client metricsV1Beta3Client = MetricsV1Beta3Client.create()) {
   GetJobMetricsRequest request =
       GetJobMetricsRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setJobId("jobId101296568")
           .setStartTime(Timestamp.newBuilder().build())
           .setLocation("location1901043637")
           .build();
   JobMetrics response = metricsV1Beta3Client.getJobMetrics(request);
 }

SnapshotsV1Beta3Client

Service Description: Provides methods to manage snapshots of Google Cloud Dataflow jobs.

Sample for SnapshotsV1Beta3Client:


 try (SnapshotsV1Beta3Client snapshotsV1Beta3Client = SnapshotsV1Beta3Client.create()) {
   GetSnapshotRequest request =
       GetSnapshotRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setSnapshotId("snapshotId-1113817601")
           .setLocation("location1901043637")
           .build();
   Snapshot response = snapshotsV1Beta3Client.getSnapshot(request);
 }

TemplatesServiceClient

Service Description: Provides a method to create Cloud Dataflow jobs from templates.

Sample for TemplatesServiceClient:


 try (TemplatesServiceClient templatesServiceClient = TemplatesServiceClient.create()) {
   CreateJobFromTemplateRequest request =
       CreateJobFromTemplateRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setJobName("jobName-1438096408")
           .putAllParameters(new HashMap<String, String>())
           .setEnvironment(RuntimeEnvironment.newBuilder().build())
           .setLocation("location1901043637")
           .build();
   Job response = templatesServiceClient.createJobFromTemplate(request);
 }

FlexTemplatesServiceClient

Service Description: Provides a service for Flex templates. This feature is not ready yet.

Sample for FlexTemplatesServiceClient:


 try (FlexTemplatesServiceClient flexTemplatesServiceClient =
     FlexTemplatesServiceClient.create()) {
   LaunchFlexTemplateRequest request =
       LaunchFlexTemplateRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setLaunchParameter(LaunchFlexTemplateParameter.newBuilder().build())
           .setLocation("location1901043637")
           .setValidateOnly(true)
           .build();
   LaunchFlexTemplateResponse response = flexTemplatesServiceClient.launchFlexTemplate(request);
 }