REST Resource: projects.locations.jobs

Resource: Job

The storage batch operations job description.

JSON representation 
{
  "name": string,
  "description": string,
  "loggingConfig": {
    object (LoggingConfig)
  },
  "createTime": string,
  "scheduleTime": string,
  "completeTime": string,
  "counters": {
    object (Counters)
  },
  "errorSummaries": [
    {
      object (ErrorSummary)
    }
  ],
  "state": enum (State),

  // Union field source can be only one of the following:
  "bucketList": {
    object (BucketList)
  }
  // End of list of possible types for union field source.

  // Union field transformation can be only one of the following:
  "putObjectHold": {
    object (PutObjectHold)
  },
  "deleteObject": {
    object (DeleteObject)
  },
  "putMetadata": {
    object (PutMetadata)
  },
  "rewriteObject": {
    object (RewriteObject)
  }
  // End of list of possible types for union field transformation.
}
Fields
name

string

Identifier. The resource name of the job.

Format: projects/{project}/locations/global/jobs/{jobId}.

For example: projects/123456/locations/global/jobs/job01.

jobId is unique in a given project for a given location. If jobId is not specified, a server-generated identifier is assigned.
description

string

Optional. A user-provided description for the job.

Maximum length: 1024 bytes when unicode-encoded.
loggingConfig

object (LoggingConfig)

Optional. Logging configuration.
createTime

string (Timestamp format)

Output only. The time that the job was created.

Uses RFC 3339, where generated output will always be Z-normalized and uses 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z", "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30".
scheduleTime

string (Timestamp format)

Output only. The time that the job was scheduled.

Uses RFC 3339, where generated output will always be Z-normalized and uses 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z", "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30".
completeTime

string (Timestamp format)

Output only. The time that the job was completed.

Uses RFC 3339, where generated output will always be Z-normalized and uses 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z", "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30".
counters

object (Counters)

Output only. Information about the progress of the job.
errorSummaries[]

object (ErrorSummary)

Output only. Summarizes errors encountered with sample error log entries.
state

enum (State)

Output only. State of the job.
Union field source. Specifies objects to be transformed. source can be only one of the following:
bucketList

object (BucketList)

Specifies a list of buckets and their objects to be transformed.
Union field transformation. Operation to be performed on the objects. transformation can be only one of the following:
putObjectHold

object (PutObjectHold)

Changes object hold status.
deleteObject

object (DeleteObject)

Delete objects.
putMetadata

object (PutMetadata)

Updates object metadata. Allows updating fixed-key and custom metadata and fixed-key metadata. For example, Cache-Control, Content-Disposition, Content-Encoding, Content-Language, Content-Type, and Custom-Time.
rewriteObject

object (RewriteObject)

Rewrites the object and updates metadata like KMS key.

BucketList

Describes list of buckets and their objects to be transformed.

JSON representation 
{
  "buckets": [
    {
      object (Bucket)
    }
  ]
}
Fields
buckets[]

object (Bucket)

Required. List of buckets and their objects to be transformed. You can specify only one bucket per job. If multiple buckets are specified, an error occurs.

Bucket

Describes configuration of a single bucket and its objects to be transformed.

JSON representation 
{
  "bucket": string,

  // Union field object_configuration can be only one of the following:
  "prefixList": {
    object (PrefixList)
  },
  "manifest": {
    object (Manifest)
  }
  // End of list of possible types for union field object_configuration.
}
Fields
bucket

string

Required. Bucket name for the objects to be transformed.
Union field object_configuration. Specifies objects to be transformed. object_configuration can be only one of the following:
prefixList

object (PrefixList)

Specifies objects matching a prefix set.
manifest

object (Manifest)

Specifies objects in a manifest file.

PrefixList

Describes prefixes of objects to be transformed.

JSON representation 
{
  "includedObjectPrefixes": [
    string
  ]
}
Fields
includedObjectPrefixes[]

string

Optional. Specify one or more object prefixes. For example:

  • To match one object, use a single prefix, prefix1.

  • To match multiple objects, use comma-separated prefixes, prefix1,prefix2.

  • To match all objects, use an empty prefix,''

Manifest

Describes list of objects to be transformed.

JSON representation 
{
  "manifestLocation": string
}
Fields
manifestLocation

string

Required. Specify the manifest file location, for example, gs://bucket_name/path/object_name.csv. The manifest is a CSV file, uploaded to Cloud Storage, that contains one object or a list of objects that you want to process. Each row in the manifest must include the bucket and name of the object. You can optionally specify the generation of the object. If you don't specify the generation, the current version of the object is used.

The file must include a header row with the following format: bucket,name,generation. The generation column is optional. For example,

bucket,name,generation
bucket_1,object_1,generation_1
bucket_1,object_2,generation_2
bucket_1,object_3,generation_3

Note: The manifest file must specify only objects within the bucket provided to the job. Rows referencing objects in other buckets are ignored.

PutObjectHold

Describes options to update object hold.

JSON representation 
{
  "temporaryHold": enum (HoldStatus),
  "eventBasedHold": enum (HoldStatus)
}
Fields
temporaryHold

enum (HoldStatus)

Required. Updates object temporary hold state. When object temporary hold is set, object cannot be deleted or replaced.
eventBasedHold

enum (HoldStatus)

Required. Updates object event based holds state. When object event based hold is set, object cannot be deleted or replaced. Resets object's time in the bucket for the purposes of the retention period.

HoldStatus

Describes the status of the hold.

Enums
HOLD_STATUS_UNSPECIFIED Default value. Object hold status is not changed.
SET Places the hold.
UNSET Releases the hold.

DeleteObject

Describes options to delete an object.

JSON representation 
{
  "permanentObjectDeletionEnabled": boolean
}
Fields
permanentObjectDeletionEnabled

boolean

Required. Controls deletion behavior when versioning is enabled for the object's bucket. If true, both live and noncurrent objects will be permanently deleted. Otherwise live objects in versioned buckets will become noncurrent and objects that were already noncurrent will be skipped. This setting doesn't have any impact on the Soft Delete feature. All objects deleted by this service can be be restored for the duration of the Soft Delete retention duration if enabled. If enabled and the manifest doesn't specify an object's generation, a GetObjectMetadata call is be made to determine the live object generation.

PutMetadata

Describes options to update object metadata.

JSON representation 
{
  "customMetadata": {
    string: string,
    ...
  },
  "contentDisposition": string,
  "contentEncoding": string,
  "contentLanguage": string,
  "contentType": string,
  "cacheControl": string,
  "customTime": string
}
Fields
customMetadata

map (key: string, value: string)

Optional. Updates the object's custom metadata. This operation adds or sets individual custom metadata key-value pairs. Keys specified with empty values will have their values cleared. Existing custom metadata keys not included in the request remain unchanged. For details, see Custom-Metadata.

An object containing a list of "key": "value" pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
contentDisposition

string

Optional. Updates objects Content-Disposition fixed metadata. Unset values in the request are ignored. To clear the metadata, set an empty value. For details, see Content-Disposition.
contentEncoding

string

Optional. Updates the objects Content-Encoding fixed metadata. Unset values in the request are ignored. To clear the metadata, set an empty value. For details, see Content-Encoding.
contentLanguage

string

Optional. Updates the objects fixed content language metadata. Metadata values must use ISO 639-1 language codes. The maximum length for metadata values is 100 characters. Unset values in the request are ignored. To clear the metadata, set an empty value. For details, see Content-Language.
contentType

string

Optional. Updates objects Content-Type fixed metadata. Unset values in the request are ignored. To clear the metadata, set an empty value. For details, see Content-Type.
cacheControl

string

Optional. Updates the objects Cache-Control fixed metadata. Unset values in the request are ignored. To clear the metadata, set an empty value. Additionally, the value for Custom-Time cannot decrease. For details, see Cache-Control.
customTime

string

Optional. Updates the object's fixed custom time metadata. Unset values in the request are ignored. To clear the metadata, set an empty value. For details, see Custom-Time.

RewriteObject

Describes options for object rewrite.

JSON representation 
{
  "kmsKey": string
}
Fields
kmsKey

string

Required. Resource name of the Cloud KMS key that is used to encrypt the object. The Cloud KMS key must be located in same location as the object. For details, see Encrypt an object with a Cloud KMS key

Format: projects/{project}/locations/{locationid}/keyRings/{keyring}/cryptoKeys/{key}

For example: projects/123456/locations/us-central1/keyRings/my-keyring/cryptoKeys/my-key. The object is rewritten and set with the specified KMS key.

LoggingConfig

Specifies the Cloud Logging behavior.

JSON representation 
{
  "logActions": [
    enum (LoggableAction)
  ],
  "logActionStates": [
    enum (LoggableActionState)
  ]
}
Fields
logActions[]

enum (LoggableAction)

Required. Specifies the actions to be logged.
logActionStates[]

enum (LoggableActionState)

Required. States in which actions are logged. If empty, no logs are generated.

LoggableAction

Loggable actions types.

Enums
LOGGABLE_ACTION_UNSPECIFIED Illegal value, to avoid allowing a default.
TRANSFORM The corresponding transform action in this job.

LoggableActionState

Loggable action states filter.

Enums
LOGGABLE_ACTION_STATE_UNSPECIFIED Illegal value, to avoid allowing a default.
SUCCEEDED LoggableAction completed successfully. SUCCEEDED actions are logged as [INFO][google.logging.type.LogSeverity.INFO].
FAILED LoggableAction terminated in an error state. FAILED actions are logged as [ERROR][google.logging.type.LogSeverity.ERROR].

Counters

Describes details about the progress of the job.

JSON representation 
{
  "totalObjectCount": string,
  "succeededObjectCount": string,
  "failedObjectCount": string
}
Fields
totalObjectCount

string (int64 format)

Output only. Number of objects listed.
succeededObjectCount

string (int64 format)

Output only. Number of objects completed.
failedObjectCount

string (int64 format)

Output only. Number of objects failed.

ErrorSummary

A summary of errors by error code, plus a count and sample error log entries.

JSON representation 
{
  "errorCode": enum (Code),
  "errorCount": string,
  "errorLogEntries": [
    {
      object (ErrorLogEntry)
    }
  ]
}
Fields
errorCode

enum (Code)

Required. The canonical error code.
errorCount

string (int64 format)

Required. Number of errors encountered per errorCode.
errorLogEntries[]

object (ErrorLogEntry)

Required. Sample error logs.

Code

Defines error codes used for handling gRPC API responses.

When multiple error codes apply, return the most specific error code. For example, prefer OUT_OF_RANGE over FAILED_PRECONDITION if both codes apply. Similarly prefer NOT_FOUND or ALREADY_EXISTS over FAILED_PRECONDITION.

Enums
OK

Returned when the operation completes successfully.

HTTP Mapping: 200 OK
CANCELLED

The operation was cancelled, typically by the caller.

HTTP Mapping: 499 Client Closed Request
UNKNOWN

Unknown error. For example, this error may be returned when a Status value received from another address space belongs to an error space that is not known in this address space. Also errors raised by APIs that do not return enough error information may be converted to this error.

HTTP Mapping: 500 Internal Server Error
INVALID_ARGUMENT

The client specified an invalid argument. Note that this differs from FAILED_PRECONDITION. INVALID_ARGUMENT indicates arguments that are problematic regardless of the state of the system (For example, a malformed file name).

HTTP Mapping: 400 Bad Request
DEADLINE_EXCEEDED

The deadline expired before the operation could complete. For operations that change the state of the system, this error may be returned even if the operation has completed successfully. For example, a successful response from a server could have been delayed long enough for the deadline to expire.

HTTP Mapping: 504 Gateway Timeout
NOT_FOUND

Some requested entity (For example, file or directory) was not found.

Note to server developers: if a request is denied for an entire class of users, such as gradual feature rollout or undocumented allowlist, NOT_FOUND may be used. If a request is denied for some users within a class of users, such as user-based access control, PERMISSION_DENIED must be used.

HTTP Mapping: 404 Not Found
ALREADY_EXISTS

The entity that a client attempted to create (For example, file or directory) already exists.

HTTP Mapping: 409 Conflict
PERMISSION_DENIED

The caller does not have permission to execute the specified operation. PERMISSION_DENIED must not be used for rejections caused by exhausting some resource (use RESOURCE_EXHAUSTED instead for those errors). PERMISSION_DENIED must not be used if the caller can not be identified (use UNAUTHENTICATED instead for those errors). This error code does not imply the request is valid or the requested entity exists or satisfies other pre-conditions.

HTTP Mapping: 403 Forbidden
UNAUTHENTICATED

The request does not have valid authentication credentials for the operation.

HTTP Mapping: 401 Unauthorized
RESOURCE_EXHAUSTED

Some resource has been exhausted, perhaps a per-user quota, or perhaps the entire file system is out of space.

HTTP Mapping: 429 Too Many Requests
FAILED_PRECONDITION

The operation was rejected because the system is not in a state required for the operation's execution. For example, the directory to be deleted is non-empty, an rmdir operation is applied to a non-directory, etc.

Service implementors can use the following guidelines to decide between FAILED_PRECONDITION, ABORTED, and UNAVAILABLE:

  • Use UNAVAILABLE if the client can retry just the failing call.
  • Use ABORTED if the client should retry at a higher level. For example, when a client-specified test-and-set fails, indicating the client should restart a read-modify-write sequence.
  • Use FAILED_PRECONDITION if the client should not retry until the system state has been explicitly fixed. For example, if an "rmdir" fails because the directory is non-empty, FAILED_PRECONDITION should be returned since the client should not retry unless the files are deleted from the directory.

HTTP Mapping: 400 Bad Request
ABORTED

The operation was aborted, typically due to a concurrency issue such as a sequencer check failure or transaction abort.

See the guidelines above for deciding between FAILED_PRECONDITION, ABORTED, and UNAVAILABLE.

HTTP Mapping: 409 Conflict
OUT_OF_RANGE

The operation was attempted past the valid range. For example, seeking or reading past end-of-file.

Unlike INVALID_ARGUMENT, this error indicates a problem that may be fixed if the system state changes. For example, a 32-bit file system will generate INVALID_ARGUMENT if asked to read at an offset that is not in the range [0,2^32-1], but it will generate OUT_OF_RANGE if asked to read from an offset past the current file size.

There is a fair bit of overlap between FAILED_PRECONDITION and OUT_OF_RANGE. We recommend using OUT_OF_RANGE (the more specific error) when it applies so that callers who are iterating through a space can easily look for an OUT_OF_RANGE error to detect when they are done.

HTTP Mapping: 400 Bad Request
UNIMPLEMENTED

The operation is not implemented or is not supported/enabled in this service.

HTTP Mapping: 501 Not Implemented
INTERNAL

Internal errors. This means that some invariants expected by the underlying system have been broken. This error code is reserved for serious errors.

HTTP Mapping: 500 Internal Server Error
UNAVAILABLE

The service is currently unavailable. This is most likely a transient condition, which can be corrected by retrying with a backoff. Note that it is not always safe to retry non-idempotent operations.

See the guidelines above for deciding between FAILED_PRECONDITION, ABORTED, and UNAVAILABLE.

HTTP Mapping: 503 Service Unavailable
DATA_LOSS

Unrecoverable data loss or corruption.

HTTP Mapping: 500 Internal Server Error

ErrorLogEntry

An entry describing an error that has occurred.

JSON representation 
{
  "objectUri": string,
  "errorDetails": [
    string
  ]
}
Fields
objectUri

string

Required. Output only. Object URL. For example, gs://my_bucket/object.txt
errorDetails[]

string

Optional. Output only. A maximum of 5 error log entries are recorded per error code for each job.

State

Describes state of a job.

Enums
STATE_UNSPECIFIED Default value. This value is unused.
RUNNING In progress.
SUCCEEDED Completed successfully.
CANCELED Cancelled by the user.
FAILED Terminated due to an unrecoverable failure.

Methods

cancel

 Cancels a batch job in a given project for a given location.

create

 Creates a batch job in a given project for a given location.

delete

 Deletes a batch job in a given project for a given location.

get

 Gets a batch job in a given project for a given location.

list

 Lists all the batch jobs in a given project for a given location.