Types overview

CheckValidCredsResponse

A response indicating whether the credentials exist and are valid.
Fields
hasValidCreds

boolean

If set to true, the credentials exist and are valid.

DataSource

Defines the properties and custom parameters for a data source.
Fields
authorizationType

enum

Indicates the type of authorization.
Enum type. Can be one of the following:
AUTHORIZATION_TYPE_UNSPECIFIED Type unspecified.
AUTHORIZATION_CODE Use OAuth 2 authorization codes that can be exchanged for a refresh token on the backend.
GOOGLE_PLUS_AUTHORIZATION_CODE Return an authorization code for a given Google+ page that can then be exchanged for a refresh token on the backend.
FIRST_PARTY_OAUTH Use First Party OAuth.
clientId

string

Data source client id which should be used to receive refresh token.
dataRefreshType

enum

Specifies whether the data source supports automatic data refresh for the past few days, and how it's supported. For some data sources, data might not be complete until a few days later, so it's useful to refresh data automatically.
Enum type. Can be one of the following:
DATA_REFRESH_TYPE_UNSPECIFIED The data source won't support data auto refresh, which is default value.
SLIDING_WINDOW The data source supports data auto refresh, and runs will be scheduled for the past few days. Does not allow custom values to be set for each transfer config.
CUSTOM_SLIDING_WINDOW The data source supports data auto refresh, and runs will be scheduled for the past few days. Allows custom values to be set for each transfer config.
dataSourceId

string

Data source id.
defaultDataRefreshWindowDays

integer (int32 format)

Default data refresh window on days. Only meaningful when data_refresh_type = SLIDING_WINDOW.
defaultSchedule

string

Default data transfer schedule. Examples of valid schedules include: 1st,3rd monday of month 15:30, every wed,fri of jan,jun 13:15, and first sunday of quarter 00:00.
description

string

User friendly data source description string.
displayName

string

User friendly data source name.
helpUrl

string

Url for the help document for this data source.
manualRunsDisabled

boolean

Disables backfilling and manual run scheduling for the data source.
minimumScheduleInterval

string (Duration format)

The minimum interval for scheduler to schedule runs.
name

string

Output only. Data source resource name.
parameters[]

object (DataSourceParameter)

Data source parameters.
scopes[]

string

Api auth scopes for which refresh token needs to be obtained. These are scopes needed by a data source to prepare data and ingest them into BigQuery, e.g., https://www.googleapis.com/auth/bigquery
supportsCustomSchedule

boolean

Specifies whether the data source supports a user defined schedule, or operates on the default schedule. When set to true, user can override default schedule.
supportsMultipleTransfers

boolean

Deprecated. This field has no effect.
transferType

enum

Deprecated. This field has no effect.
Enum type. Can be one of the following:
TRANSFER_TYPE_UNSPECIFIED Invalid or Unknown transfer type placeholder.
BATCH Batch data transfer.
STREAMING Streaming data transfer. Streaming data source currently doesn't support multiple transfer configs per project.
updateDeadlineSeconds

integer (int32 format)

The number of seconds to wait for an update from the data source before the Data Transfer Service marks the transfer as FAILED.

DataSourceParameter

A parameter used to define custom fields in a data source definition.
Fields
allowedValues[]

string

All possible values for the parameter.
deprecated

boolean

If true, it should not be used in new transfers, and it should not be visible to users.
description

string

Parameter description.
displayName

string

Parameter display name in the user interface.
fields[]

object (DataSourceParameter)

Deprecated. This field has no effect.
immutable

boolean

Cannot be changed after initial creation.
maxValue

number (double format)

For integer and double values specifies maximum allowed value.
minValue

number (double format)

For integer and double values specifies minimum allowed value.
paramId

string

Parameter identifier.
recurse

boolean

Deprecated. This field has no effect.
repeated

boolean

Deprecated. This field has no effect.
required

boolean

Is parameter required.
type

enum

Parameter type.
Enum type. Can be one of the following:
TYPE_UNSPECIFIED Type unspecified.
STRING String parameter.
INTEGER Integer parameter (64-bits). Will be serialized to json as string.
DOUBLE Double precision floating point parameter.
BOOLEAN Boolean parameter.
RECORD Deprecated. This field has no effect.
PLUS_PAGE Page ID for a Google+ Page.
validationDescription

string

Description of the requirements for this field, in case the user input does not fulfill the regex pattern or min/max values.
validationHelpUrl

string

URL to a help document to further explain the naming requirements.
validationRegex

string

Regular expression which can be used for parameter validation.

EmailPreferences

Represents preferences for sending email notifications for transfer run events.
Fields
enableFailureEmail

boolean

If true, email notifications will be sent on transfer run failures.

EnrollDataSourcesRequest

A request to enroll a set of data sources so they are visible in the BigQuery UI's Transfer tab.
Fields
dataSourceIds[]

string

Data sources that are enrolled. It is required to provide at least one data source id.

ListDataSourcesResponse

Returns list of supported data sources and their metadata.
Fields
dataSources[]

object (DataSource)

List of supported data sources and their transfer settings.
nextPageToken

string

Output only. The next-pagination token. For multiple-page list results, this token can be used as the ListDataSourcesRequest.page_token to request the next page of list results.

ListLocationsResponse

The response message for Locations.ListLocations.
Fields
locations[]

object (Location)

A list of locations that matches the specified filter in the request.
nextPageToken

string

The standard List next-page token.

ListTransferConfigsResponse

The returned list of pipelines in the project.
Fields
nextPageToken

string

Output only. The next-pagination token. For multiple-page list results, this token can be used as the ListTransferConfigsRequest.page_token to request the next page of list results.
transferConfigs[]

object (TransferConfig)

Output only. The stored pipeline transfer configurations.

ListTransferLogsResponse

The returned list transfer run messages.
Fields
nextPageToken

string

Output only. The next-pagination token. For multiple-page list results, this token can be used as the GetTransferRunLogRequest.page_token to request the next page of list results.
transferMessages[]

object (TransferMessage)

Output only. The stored pipeline transfer messages.

ListTransferRunsResponse

The returned list of pipelines in the project.
Fields
nextPageToken

string

Output only. The next-pagination token. For multiple-page list results, this token can be used as the ListTransferRunsRequest.page_token to request the next page of list results.
transferRuns[]

object (TransferRun)

Output only. The stored pipeline transfer runs.

Location

A resource that represents Google Cloud Platform location.
Fields
displayName

string

The friendly name for this location, typically a nearby city name. For example, "Tokyo".
labels

map (key: string, value: string)

Cross-service attributes for the location. For example {"cloud.googleapis.com/region": "us-east1"}
locationId

string

The canonical id for this location. For example: "us-east1".
metadata

map (key: string, value: any)

Service-specific metadata. For example the available capacity at the given location.
name

string

Resource name for the location, which may vary between implementations. For example: "projects/example-project/locations/us-east1"

ScheduleOptions

Options customizing the data transfer schedule.
Fields
disableAutoScheduling

boolean

If true, automatic scheduling of data transfer runs for this configuration will be disabled. The runs can be started on ad-hoc basis using StartManualTransferRuns API. When automatic scheduling is disabled, the TransferConfig.schedule field will be ignored.
endTime

string (Timestamp format)

Defines time to stop scheduling transfer runs. A transfer run cannot be scheduled at or after the end time. The end time can be changed at any moment. The time when a data transfer can be triggered manually is not limited by this option.
startTime

string (Timestamp format)

Specifies time to start scheduling transfer runs. The first run will be scheduled at or after the start time according to a recurrence pattern defined in the schedule string. The start time can be changed at any moment. The time when a data transfer can be triggered manually is not limited by this option.

ScheduleTransferRunsRequest

A request to schedule transfer runs for a time range.
Fields
endTime

string (Timestamp format)

Required. End time of the range of transfer runs. For example, "2017-05-30T00:00:00+00:00".
startTime

string (Timestamp format)

Required. Start time of the range of transfer runs. For example, "2017-05-25T00:00:00+00:00".

ScheduleTransferRunsResponse

A response to schedule transfer runs for a time range.
Fields
runs[]

object (TransferRun)

The transfer runs that were scheduled.

StartManualTransferRunsRequest

A request to start manual transfer runs.
Fields
requestedRunTime

string (Timestamp format)

Specific run_time for a transfer run to be started. The requested_run_time must not be in the future.
requestedTimeRange

object (TimeRange)

Time range for the transfer runs that should be started.

StartManualTransferRunsResponse

A response to start manual transfer runs.
Fields
runs[]

object (TransferRun)

The transfer runs that were created.

Status

The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC. Each Status message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the API Design Guide.
Fields
code

integer (int32 format)

The status code, which should be an enum value of google.rpc.Code.
details[]

object

A list of messages that carry the error details. There is a common set of message types for APIs to use.
message

string

A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.

TimeRange

A specification for a time range, this will request transfer runs with run_time between start_time (inclusive) and end_time (exclusive).
Fields
endTime

string (Timestamp format)

End time of the range of transfer runs. For example, "2017-05-30T00:00:00+00:00". The end_time must not be in the future. Creates transfer runs where run_time is in the range between start_time (inclusive) and end_time (exclusive).
startTime

string (Timestamp format)

Start time of the range of transfer runs. For example, "2017-05-25T00:00:00+00:00". The start_time must be strictly less than the end_time. Creates transfer runs where run_time is in the range between start_time (inclusive) and end_time (exclusive).

TransferConfig

Represents a data transfer configuration. A transfer configuration contains all metadata needed to perform a data transfer. For example, destination_dataset_id specifies where data should be stored. When a new transfer configuration is created, the specified destination_dataset_id is created when needed and shared with the appropriate data source service account.
Fields
dataRefreshWindowDays

integer (int32 format)

The number of days to look back to automatically refresh the data. For example, if data_refresh_window_days = 10, then every day BigQuery reingests data for [today-10, today-1], rather than ingesting data for just [today-1]. Only valid if the data source supports the feature. Set the value to 0 to use the default value.
dataSourceId

string

Data source id. Cannot be changed once data transfer is created.
datasetRegion

string

Output only. Region in which BigQuery dataset is located.
destinationDatasetId

string

The BigQuery target dataset id.
disabled

boolean

Is this config disabled. When set to true, no runs are scheduled for a given transfer.
displayName

string

User specified display name for the data transfer.
emailPreferences

object (EmailPreferences)

Email notifications will be sent according to these preferences to the email address of the user who owns this transfer config.
name

string

The resource name of the transfer config. Transfer config names have the form projects/{project_id}/locations/{region}/transferConfigs/{config_id}. Where config_id is usually a uuid, even though it is not guaranteed or required. The name is ignored when creating a transfer config.
nextRunTime

string (Timestamp format)

Output only. Next time when data transfer will run.
notificationPubsubTopic

string

Pub/Sub topic where notifications will be sent after transfer runs associated with this transfer config finish. The format for specifying a pubsub topic is: projects/{project}/topics/{topic}
ownerInfo

object (UserInfo)

Output only. Information about the user whose credentials are used to transfer data. Populated only for transferConfigs.get requests. In case the user information is not available, this field will not be populated.
params

map (key: string, value: any)

Parameters specific to each data source. For more information see the bq tab in the 'Setting up a data transfer' section for each data source. For example the parameters for Cloud Storage transfers are listed here: https://cloud.google.com/bigquery-transfer/docs/cloud-storage-transfer#bq
schedule

string

Data transfer schedule. If the data source does not support a custom schedule, this should be empty. If it is empty, the default value for the data source will be used. The specified times are in UTC. Examples of valid format: 1st,3rd monday of month 15:30, every wed,fri of jan,jun 13:15, and first sunday of quarter 00:00. See more explanation about the format here: https://cloud.google.com/appengine/docs/flexible/python/scheduling-jobs-with-cron-yaml#the_schedule_format NOTE: The minimum interval time between recurring transfers depends on the data source; refer to the documentation for your data source.
scheduleOptions

object (ScheduleOptions)

Options customizing the data transfer schedule.
state

enum

Output only. State of the most recently updated transfer run.
Enum type. Can be one of the following:
TRANSFER_STATE_UNSPECIFIED State placeholder (0).
PENDING Data transfer is scheduled and is waiting to be picked up by data transfer backend (2).
RUNNING Data transfer is in progress (3).
SUCCEEDED Data transfer completed successfully (4).
FAILED Data transfer failed (5).
CANCELLED Data transfer is cancelled (6).
updateTime

string (Timestamp format)

Output only. Data transfer modification time. Ignored by server on input.
userId

string (int64 format)

Deprecated. Unique ID of the user on whose behalf transfer is done.

TransferMessage

Represents a user facing message for a particular data transfer run.
Fields
messageText

string

Message text.
messageTime

string (Timestamp format)

Time when message was logged.
severity

enum

Message severity.
Enum type. Can be one of the following:
MESSAGE_SEVERITY_UNSPECIFIED No severity specified.
INFO Informational message.
WARNING Warning message.
ERROR Error message.

TransferRun

Represents a data transfer run.
Fields
dataSourceId

string

Output only. Data source id.
destinationDatasetId

string

Output only. The BigQuery target dataset id.
emailPreferences

object (EmailPreferences)

Output only. Email notifications will be sent according to these preferences to the email address of the user who owns the transfer config this run was derived from.
endTime

string (Timestamp format)

Output only. Time when transfer run ended. Parameter ignored by server for input requests.
errorStatus

object (Status)

Status of the transfer run.
name

string

The resource name of the transfer run. Transfer run names have the form projects/{project_id}/locations/{location}/transferConfigs/{config_id}/runs/{run_id}. The name is ignored when creating a transfer run.
notificationPubsubTopic

string

Output only. Pub/Sub topic where a notification will be sent after this transfer run finishes. The format for specifying a pubsub topic is: projects/{project}/topics/{topic}
params

map (key: string, value: any)

Output only. Parameters specific to each data source. For more information see the bq tab in the 'Setting up a data transfer' section for each data source. For example the parameters for Cloud Storage transfers are listed here: https://cloud.google.com/bigquery-transfer/docs/cloud-storage-transfer#bq
runTime

string (Timestamp format)

For batch transfer runs, specifies the date and time of the data should be ingested.
schedule

string

Output only. Describes the schedule of this transfer run if it was created as part of a regular schedule. For batch transfer runs that are scheduled manually, this is empty. NOTE: the system might choose to delay the schedule depending on the current load, so schedule_time doesn't always match this.
scheduleTime

string (Timestamp format)

Minimum time after which a transfer run can be started.
startTime

string (Timestamp format)

Output only. Time when transfer run was started. Parameter ignored by server for input requests.
state

enum

Data transfer run state. Ignored for input requests.
Enum type. Can be one of the following:
TRANSFER_STATE_UNSPECIFIED State placeholder (0).
PENDING Data transfer is scheduled and is waiting to be picked up by data transfer backend (2).
RUNNING Data transfer is in progress (3).
SUCCEEDED Data transfer completed successfully (4).
FAILED Data transfer failed (5).
CANCELLED Data transfer is cancelled (6).
updateTime

string (Timestamp format)

Output only. Last time the data transfer run state was updated.
userId

string (int64 format)

Deprecated. Unique ID of the user on whose behalf transfer is done.

UserInfo

Information about a user.
Fields
email

string

E-mail address of the user.