A Dataproc job for running Apache Pig queries on YARN.
JSON representation
{"continueOnFailure": boolean,"scriptVariables": {string: string,...},"properties": {string: string,...},"jarFileUris": [string],"loggingConfig": {object (LoggingConfig)},// Union field queries can be only one of the following:"queryFileUri": string,"queryList": {object (QueryList)}// End of list of possible types for union field queries.}
Fields
continueOnFailure
boolean
Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
scriptVariables
map (key: string, value: string)
Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
properties
map (key: string, value: string)
Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
jarFileUris[]
string
Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Optional. The runtime log config for job execution.
Union field queries. Required. The sequence of Pig queries to execute, specified as an HCFS file URI or a list of queries. queries can be only one of the following:
queryFileUri
string
The HCFS URI of the script that contains the Pig queries.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-02-27 UTC."],[[["This document describes the JSON representation of a Dataproc job for running Apache Pig queries on YARN, detailing its structure and fields."],["The `queries` field is required and can be specified as either a URI to a file containing Pig queries (`queryFileUri`) or a list of queries (`queryList`)."],["The job configuration supports optional fields like `continueOnFailure` (to dictate behavior on query failure), `scriptVariables` (for Pig variable substitution), `properties` (for Pig configuration), `jarFileUris` (for including JAR files), and `loggingConfig` (for runtime log configuration)."],["`scriptVariables` and `properties` utilize a map structure to assign strings as key-value pairs, allowing for query variable and Pig property settings respectively."]]],[]]
