Package Classes (2.26.0)

Client for interacting with BigQuery Storage API.

The BigQuery storage API can be used to read data stored in BigQuery.

BigQuery Write API.

The Write API can be used to write data to BigQuery.

For supplementary information about the Write API, see:

https://cloud.google.com/bigquery/docs/write-api

An iterable of rows from a read session.

An iterator of rows from a read session message.

A stream of results from a read rows request.

This stream is an iterable of ReadRowsResponse. Iterate over it to fetch all row messages.

If the fastavro library is installed, use the rows() method to parse all messages into a stream of row dictionaries.

If the pandas and fastavro libraries are installed, use the to_dataframe() method to parse all messages into a pandas.DataFrame.

This object should not be created directly, but is returned by other methods in this library.

BigQuery Read API.

The Read API can be used to read data from BigQuery.

BigQuery Read API.

The Read API can be used to read data from BigQuery.

BigQuery Write API.

The Write API can be used to write data to BigQuery.

For supplementary information about the Write API, see:

https://cloud.google.com/bigquery/docs/write-api

BigQuery Write API.

The Write API can be used to write data to BigQuery.

For supplementary information about the Write API, see:

https://cloud.google.com/bigquery/docs/write-api

Request message for AppendRows.

Because AppendRows is a bidirectional streaming RPC, certain parts of the AppendRowsRequest need only be specified for the first request before switching table destinations. You can also switch table destinations within the same connection for the default stream.

The size of a single AppendRowsRequest must be less than 10 MB in size. Requests larger than this return an error, typically INVALID_ARGUMENT.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

An enum to indicate how to interpret missing values of fields that are present in user schema but missing in rows. A missing value can represent a NULL or a column default value defined in BigQuery table schema.

The abstract base class for a message.

ProtoData contains the data rows and schema when constructing append requests.

Response message for AppendRows.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

AppendResult is returned for successful append requests.

Arrow RecordBatch.

Arrow schema as specified in https://arrow.apache.org/docs/python/api/datatypes.html and serialized to bytes using IPC:

https://arrow.apache.org/docs/format/Columnar.html#serialization-and-interprocess-communication-ipc

See code samples on how this message can be deserialized.

Contains options specific to Arrow Serialization.

Compression codec's supported by Arrow.

Avro rows.

Avro schema.

Contains options specific to Avro Serialization.

Request message for BatchCommitWriteStreams.

Response message for BatchCommitWriteStreams.

Request message for CreateReadSession.

Request message for CreateWriteStream.

Data format for input or output data.

Request message for invoking FinalizeWriteStream.

Response message for FinalizeWriteStream.

Request message for FlushRows.

Respond message for FlushRows.

Request message for GetWriteStreamRequest.

ProtoSchema describes the schema of the serialized protocol buffer data rows.

Request message for ReadRows.

Response from calling ReadRows may include row data, progress and throttling information.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Information about the ReadSession.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Additional attributes when reading a table.

Options dictating how we read a table.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Specifies which compression codec to attempt on the entire serialized response payload (either Arrow record batch or Avro rows). This is not to be confused with the Apache Arrow native compression codecs specified in ArrowSerializationOptions. For performance reasons, when creating a read session requesting Arrow responses, setting both native Arrow compression and application-level response compression will not be allowed - choose, at most, one kind of compression.

Information about a single stream that gets data out of the storage system. Most of the information about ReadStream instances is aggregated, making ReadStream lightweight.

The message that presents row level error info in a request.

Error code for RowError.

Request message for SplitReadStream.

Response message for SplitReadStream.

Structured custom BigQuery Storage error message. The error can be attached as error details in the returned rpc Status. In particular, the use of error codes allows more structured error handling, and reduces the need to evaluate unstructured error text strings.

Error code for StorageError.

Estimated stream statistics for a given read Stream.

TableFieldSchema defines a single field/column within a table schema.

Represents the type of a field element.

Schema of a table. This schema is a subset of google.cloud.bigquery.v2.TableSchema containing information necessary to generate valid message to write to BigQuery.

Information on if the current connection is being throttled.

Information about a single stream that gets data inside the storage system.

Type enum of the stream.

Mode enum of the stream.

WriteStreamView is a view enum that controls what details about a write stream should be returned.

BigQuery Metastore Partition Service API. This service is used for managing metastore partitions in BigQuery metastore. The service supports only batch operations for write.

BigQuery Metastore Partition Service API. This service is used for managing metastore partitions in BigQuery metastore. The service supports only batch operations for write.

Request message for BatchCreateMetastorePartitions.

Response message for BatchCreateMetastorePartitions.

Request message for BatchDeleteMetastorePartitions. The MetastorePartition is uniquely identified by values, which is an ordered list. Hence, there is no separate name or partition id field.

Structured custom error message for batch size too large error. The error can be attached as error details in the returned rpc Status for more structured error handling in the client.

Request message for BatchUpdateMetastorePartitions.

Response message for BatchUpdateMetastorePartitions.

Request message for CreateMetastorePartition. The MetastorePartition is uniquely identified by values, which is an ordered list. Hence, there is no separate name or partition id field.

Schema description of a metastore partition column.

Request message for ListMetastorePartitions.

Response message for ListMetastorePartitions.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Information about a Hive partition.

The abstract base class for a message.

List of metastore partitions.

Represents the values of a metastore partition.

Information about a single stream that is used to read partitions.

Serializer and deserializer information.

The abstract base class for a message.

Contains information about the physical storage of the data in the metastore partition.

List of streams.

The top-level message sent by the client to the [Partitions.StreamMetastorePartitions][] method. Follows the default gRPC streaming maximum size of 4 MB.

This is the response message sent by the server to the client for the [Partitions.StreamMetastorePartitions][] method when the commit is successful. Server will close the stream after sending this message.

Request message for UpdateMetastorePartition.

Client for interacting with BigQuery Storage API.

The BigQuery storage API can be used to read data stored in BigQuery.

BigQuery Write API.

The Write API can be used to write data to BigQuery.

The google.cloud.bigquery.storage.v1 API </bigquery/docs/reference/storage/rpc/google.cloud.bigquery.storage.v1>__ should be used instead of the v1beta2 API for BigQueryWrite operations.

BigQuery Read API.

The Read API can be used to read data from BigQuery.

New code should use the v1 Read API going forward, if they don't use Write API at the same time.

BigQuery Read API.

The Read API can be used to read data from BigQuery.

New code should use the v1 Read API going forward, if they don't use Write API at the same time.

BigQuery Write API.

The Write API can be used to write data to BigQuery.

The google.cloud.bigquery.storage.v1 API </bigquery/docs/reference/storage/rpc/google.cloud.bigquery.storage.v1>__ should be used instead of the v1beta2 API for BigQueryWrite operations.

BigQuery Write API.

The Write API can be used to write data to BigQuery.

The google.cloud.bigquery.storage.v1 API </bigquery/docs/reference/storage/rpc/google.cloud.bigquery.storage.v1>__ should be used instead of the v1beta2 API for BigQueryWrite operations.

Request message for AppendRows.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Proto schema and data.

Response message for AppendRows.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

AppendResult is returned for successful append requests.

Arrow RecordBatch.

Arrow schema as specified in https://arrow.apache.org/docs/python/api/datatypes.html and serialized to bytes using IPC:

https://arrow.apache.org/docs/format/Columnar.html#serialization-and-interprocess-communication-ipc

See code samples on how this message can be deserialized.

Contains options specific to Arrow Serialization.

The IPC format to use when serializing Arrow streams.

Avro rows.

Avro schema.

Request message for BatchCommitWriteStreams.

Response message for BatchCommitWriteStreams.

Request message for CreateReadSession.

Request message for CreateWriteStream.

Data format for input or output data.

Request message for invoking FinalizeWriteStream.

Response message for FinalizeWriteStream.

Request message for FlushRows.

Respond message for FlushRows.

Request message for GetWriteStreamRequest.

ProtoSchema describes the schema of the serialized protocol buffer data rows.

Request message for ReadRows.

Response from calling ReadRows may include row data, progress and throttling information.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Information about the ReadSession.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Additional attributes when reading a table.

Options dictating how we read a table.

Information about a single stream that gets data out of the storage system. Most of the information about ReadStream instances is aggregated, making ReadStream lightweight.

Request message for SplitReadStream.

Structured custom BigQuery Storage error message. The error can be attached as error details in the returned rpc Status. In particular, the use of error codes allows more structured error handling, and reduces the need to evaluate unstructured error text strings.

Error code for StorageError.

Estimated stream statistics for a given Stream.

A field in TableSchema

Schema of a table

Information on if the current connection is being throttled.

Information about a single stream that gets data inside the storage system.

Type enum of the stream.

Encapsulation of the asynchronous execution of an action.

This object is returned from long-running BigQuery Storage API calls, and is the interface to determine the status of those calls.

This object should not be created directly, but is returned by other methods in this library.

A manager object which can append rows to a stream.

Parent client for calling the Cloud BigQuery Storage API.

This is the base from which all interactions with the API occur.

API documentation for bigquery_storage_v1.reader module.

Parent client for calling the Cloud BigQuery Storage API.

This is the base from which all interactions with the API occur.

API documentation for bigquery_storage_v1beta2.writer module.