Release history for google-cloud-bigquery

1.51.1 (2024-12-13)

Bug Fixes

• Allow inserts to a target dataset/table in another project (#28097)

1.51.0 (2024-12-04)

Features

• load jobs with dest tables in another project (#27681)

  Bug Fixes

• remove reload! call on patch_table (#27684)

1.50.0 (2024-06-27)

Features

• add method in load_job to set column name character map (#26135)

1.49.1 (2024-06-05)

Bug Fixes

• Added bigdecimal to the gem dependencies (#26012)

1.49.0 (2024-03-10)

Features

• Update minimum supported Ruby version to 2.7 (#25298)

1.48.1 (2024-02-10)

Documentation

• Correct an example snippet related to storage_billing_model (#24796)

1.48.0 (2024-02-08)

Features

• Support accessing storage_ billing_model field in a dataset (#24755)

1.47.0 (2024-02-05)

Features

• add json type support (#24491)

1.46.1 (2024-01-26)

Bug Fixes

• Raise an error on mismatching universe domain (#24485)

1.46.0 (2024-01-25)

Features

• Support for universe_domain (#24448)

1.45.0 (2023-09-25)

Features

• support load job with session (#23320)

1.44.2 (2023-09-12)

Bug Fixes

• Avoid dataset reload when accessing location (#22905)

1.44.1 (2023-09-08)

Bug Fixes

• remove unnecessary warning (#22904)

1.44.0 (2023-09-04)

Features

• support BIGQUERY_EMULATOR_HOST env variable for endpoint

1.43.1 (2023-05-19)

Bug Fixes

• ensure schema for data parsing (#21616)

1.43.0 (2023-05-10)

Features

• Added support for default value expression (#21540)

1.42.0 (2023-01-15)

Features

• Added support for authorized dataset (#19442)
• Added support for tags in dataset (#19350)

1.41.0 (2023-01-05)

Features

• Add support for partial projection of table metadata

  Bug Fixes

• Fix querying of array of structs in named parameters (#19466)

1.40.0 (2022-12-14)

Features

• support table snapshot and clone (#19354)

1.39.0 (2022-07-27)

Features

• Update minimum Ruby version to 2.6 (#18871)

1.38.1 / 2022-01-13

Bug Fixes

• Update Bigquery::Data#ddl? to support ALTER TABLE
• Update Bigquery::QueryJob#ddl? to support ALTER TABLE
• Remove newline character ('\n') from #ddl? statement types

Documentation

• Update Contributing docs

1.38.0 / 2021-11-16

Features

• Add session support
  • Add create_session and session_id params to Project#query_job
  • Add create_session and session_id params to Dataset#query_job
  • Add session_id param to Project#query
  • Add session_id param to Dataset#query
  • Add Job#session_id
  • Add QueryJob::Updater#create_session=
  • Add QueryJob::Updater#session_id=

1.37.0 / 2021-10-21

Features

• Add support for Avro options to external data sources
  • Add External::AvroSource

1.36.0 / 2021-09-22

Features

• Add Job#delete

Bug Fixes

• Add precision and scale to Field#add_field

1.35.1 / 2021-09-14

Documentation

• Wrap character class regex in backticks

1.35.0 / 2021-08-12

Features

• Add GEOGRAPHY schema helpers
  • Add LoadJob#geography
  • Add Schema::Field#geography
  • Add Table::Updater#geography
• Add support for GEOGRAPHY type
  • Add Schema#geography
• Add support for multistatement transaction statistics in jobs
  • Add Job#transaction_id

1.34.0 / 2021-07-20

Features

• Add DmlStatistics
  • Add QueryJob#deleted_row_count
  • Add QueryJob#inserted_row_count
  • Add QueryJob#updated_row_count
  • Add Data#deleted_row_count
  • Add Data#inserted_row_count
  • Add Data#updated_row_count

1.33.0 / 2021-07-14

Features

• Add policy tag support (Column ACLs)
  • Add policy_tags to LoadJob field helper methods
  • Add policy_tags to Schema field helper methods
  • Add policy_tags to Schema::Field field helper methods
  • Add policy_tags to Table field helper methods
  • Add Schema::Field#policy_tags
  • Add Schema::Field#policy_tags=
• Add support for parameterized types
  • Add max_length to LoadJob::Updater#bytes
  • Add max_length to LoadJob::Updater#string
  • Add max_length to Schema#bytes
  • Add max_length to Schema#string
  • Add max_length to Schema::Field#bytes
  • Add max_length to Schema::Field#string
  • Add max_length to Table::Updater#bytes
  • Add max_length to Table::Updater#string
  • Add precision and scale to LoadJob::Updater#bignumeric
  • Add precision and scale to LoadJob::Updater#numeric
  • Add precision and scale to Schema#bignumeric
  • Add precision and scale to Schema#numeric
  • Add precision and scale to Schema::Field#bignumeric
  • Add precision and scale to Schema::Field#numeric
  • Add precision and scale to Table::Updater#bignumeric
  • Add precision and scale to Table::Updater#numeric
  • Add Schema::Field#max_length
  • Add Schema::Field#precision
  • Add Schema::Field#scale

1.32.1 / 2021-07-08

Documentation

• Update AUTHENTICATION.md in handwritten packages

1.32.0 / 2021-06-21

Features

• Add support for Parquet options
  • feat(bigquery): Add Bigquery::External::ParquetSource
  • Add Parquet options to LoadJob
    • Add LoadJob#parquet_options?
    • Add LoadJob#parquet_enable_list_inference?
    • Add LoadJob#parquet_enum_as_string?
    • Add LoadJob::Updater#parquet_enable_list_inference=
    • Add LoadJob::Updater#parquet_enum_as_string=

Bug Fixes

• Expand googleauth dependency to support future 1.x versions

1.31.0 / 2021-04-28

Features

• Add support for mutable clustering configuration
  • Add Table#clustering_fields=

1.30.0 / 2021-04-20

Features

• Add support for BIGNUMERIC data type
  • Add support for BIGNUMERIC data insert.
  • Add support for BIGNUMERIC query params.
  • Add #bignumeric to Schema, Table, LoadJob::Updater and Field.
  • Update types tables in docs.

Bug Fixes

• Fix Table#time_partitioning_expiration=
  • Accept nil argument.

1.29.0 / 2021-03-10

Features

• Drop support for Ruby 2.4 and add support for Ruby 3.0

1.28.0 / 2021-03-09

Features

• Add Materialized View support
  • Add Dataset#create_materialized_view
  • Add Table#materialized_view?
  • Add Table#enable_refresh?
  • Add Table#enable_refresh=
  • Add Table#last_refresh_time
  • Add Table#refresh_interval_ms
  • Add Table#refresh_interval_ms=

1.27.0 / 2021-02-10

Features

• Add Job#reservation_usage
• Add Routine#determinism_level
  • Add Routine#determinism_level
  • Add Routine#determinism_level=
  • Add Routine#determinism_level_deterministic?
  • Add Routine#determinism_level_not_deterministic?
  • Add Routine::Updater#determinism_level=

1.26.0 / 2021-01-13

Features

• Add support for Hive Partitioning
  • Add hive partitioning options to External::DataSource
  • Add hive partitioning options to LoadJob and LoadJob::Updater
• Replace google-api-client with google-apis-bigquery_v2

1.25.0 / 2020-11-16

Features

• Add routine (UDF) to Dataset::Access
• Add support for Table ACLS (IAM Policy)
  • feat(bigquery): Add support for Table ACLS
  • Add Bigquery::Policy
  • Add Table#policy
  • Add Table#test_iam_permissions
  • Add Table#update_policy

1.24.0 / 2020-10-29

Features

• Add iamMember to Dataset::Access

Bug Fixes

• Ensure dense encoding of JSON responses
  • Set query param prettyPrint=false for all requests.
  • Upgrade google-api-client to ~> 0.47

Documentation

• Update supported types for time partition type

1.23.0 / 2020-09-17

Features

• quota_project can be set via library configuration (#7627)

1.22.0 / 2020-09-10

Features

• Add support for ML model export
  • Add model support to Project#extract and #extract_job
  • Add ExtractJob#model?
  • Add ExtractJob#ml_tf_saved_model?
  • Add ExtractJob#ml_xgboost_booster?
  • Add Model#extract and #extract_job

1.21.2 / 2020-07-21

Documentation

• Update Data#each samples

1.21.1 / 2020-05-28

Documentation

• Fix a few broken links

1.21.0 / 2020-03-31

Features

• Add Job#parent_job_id and Job#script_statistics
  • Add parent_job to Project#jobs
  • Add Job#num_child_jobs
  • Add Job#parent_job_id
  • Add Job#script_statistics

1.20.0 / 2020-03-11

Features

• Add Range Partitioning
  • Add range partitioning methods to Table and Table::Updater
  • Add range partitioning methods to LoadJob
  • Add range partitioning methods to QueryJob

1.19.0 / 2020-02-11

Features

• Add Routine
  • Add Dataset#create_routine
  • Add Argument
  • Update StandardSql classes to expose public initializer
  • Add Data#ddl_target_routine and QueryJob#ddl_target_routine
• Allow row inserts to skip insert_id generation
  • Streaming inserts using an insert_id are not able to be inserted as fast as inserts without an insert_id
  • Add the ability for users to skip insert_id generation in order to speed up the inserts
  • The default behavior continues to generate insert_id values for each row inserted
  • Add yield documentation for Dataset#insert

1.18.1 / 2019-12-18

Bug Fixes

• Fix MonitorMixin usage on Ruby 2.7
  • Ruby 2.7 will error if new_cond is called before super().
  • Make the call to super() be the first call in initialize

1.18.0 / 2019-11-06

Features

• Add optional query parameter types
  • Allow query parameters to be nil/NULL when providing an optional
  • Add types argument to the following methods:
    • Project#query
    • Project#query_job
    • Dataset#query
    • Dataset#query_job
• Add param types helper methods
  • Return the BigQuery field type code, using the same format as the
  • Add Schema::Field#param_type
  • Add Schema#param_types
  • Add Data#param_types
  • Add Table#param_types
  • Add External::CvsSource#param_types
  • Add External::JsonSource#param_types
• Add support for all_users special role in Dataset access

1.17.0 / 2019-10-29

This release requires Ruby 2.4 or later.

Documentation

• Clarify which Google Cloud Platform environments support automatic authentication

1.16.0 / 2019-10-03

Features

• Add Dataset default_encryption
  • Add Dataset#default_encryption
  • Add Dataset#default_encryption=

1.15.0 / 2019-09-30

Features

• Add Model encryption
  • Add Model#encryption
  • Add Model#encryption=
• Add range support for Google Sheets
  • Add External::SheetsSource#range
  • Add External::SheetsSource#range=
• Support use_avro_logical_types on extract jobs
  • Add ExtractJob#use_avro_logical_types?
  • Add ExtractJob::Updater#use_avro_logical_types=

1.14.1 / 2019-09-04

Documentation

• Add note about streaming insert issues
  • Acknowledge tradeoffs when inserting rows soon after table metadata has been changed.
  • Add link to BigQuery Troubleshooting guide.

1.14.0 / 2019-08-23

Features

• Support overriding of service endpoint

Performance Improvements

• Use MiniMime to detect content types

Documentation

• Update documentation

1.13.0 / 2019-07-31

• Add Table#require_partition_filter
• List jobs using min and max created_at
• Reduce thread usage at startup
  • Allocate threads in pool as needed, not all up front
• Update documentation links

1.12.0 / 2019-07-10

• Add BigQuery Model API
  • Add Model
  • Add StandardSql Field, DataType, StructType
  • Add Dataset#model and Dataset#models
• Correct Float value conversion
  • Ensure that NaN, Infinity, and -Infinity are converted correctly.

1.11.2 / 2019-06-11

• Update "Loading data" link

1.11.1 / 2019-05-21

• Declare explicit dependency on mime-types

1.11.0 / 2019-02-01

• Make use of Credentials#project_id
  • Use Credentials#project_id If a project_id is not provided, use the value on the Credentials object. This value was added in googleauth 0.7.0.
  • Loosen googleauth dependency Allow for new releases up to 0.10. The googleauth devs have committed to maintaining the current API and will not make backwards compatible changes before 0.10.

1.10.0 / 2018-12-06

• Add dryrun param to Project#query_job and Dataset#query_job
• Add copy and extract methods to Project
  • Add Project#extract and Project#extract_job
  • Add Project#copy and Project#copy_job
  • Deprecate dryrun param in Table#copy_job, Table#extract_job and Table#load_job
• Fix memoization in Dataset#exists? and Table#exists?
  • Add force param to Dataset#exists? and Table#exists?

1.9.0 / 2018-10-25

• Add clustering fields to LoadJob, QueryJob and Table
• Add DDL/DML support
  • Update QueryJob#data to not return table rows for DDL/DML
  • Add DDL/DML statistics attrs to QueryJob and Data
• Add #numeric to Table::Updater and LoadJob::Updater (@leklund)

1.8.2 / 2018-09-20

• Update documentation.
  • Change documentation URL to googleapis GitHub org.
• Fix circular require warning.

1.8.1 / 2018-09-12

• Add missing documentation files to package.

1.8.0 / 2018-09-10

• Add support for OCR format.
• Update documentation.

1.7.1 / 2018-08-21

• Update documentation.

1.7.0 / 2018-06-29

• Add #schema_update_options to LoadJob and #schema_update_options= to LoadJob::Updater.
• Add time partitioning for the target table to LoadJob and QueryJob.
• Add #statement_type, #ddl_operation_performed, #ddl_target_table to QueryJob.

1.6.0 / 2018-06-22

• Documentation updates.
• Updated dependencies.

1.5.0 / 2018-05-21

• Add Schema.load and Schema.dump to read/write a table schema from/to a JSON file or other IO source. The JSON file schema is the same as for the bq CLI.
• Add support for the NUMERIC data type.
• Add documentation for enabling logging.

1.4.0 / 2018-05-07

• Add Parquet support to #load and #load_job.

1.3.0 / 2018-04-05

• Add insert_ids option to #insert in Dataset, Table, and AsyncInserter.
• Add BigQuery Project#service_account_email.
• Add support for setting Job location to nil in blocks for Job properties.

1.2.0 / 2018-03-31

• Add geo-regionalization (location) support to Jobs.
• Add Project#encryption support to Jobs.
• Rename Encryption to EncryptionConfiguration.
• Add blocks for setting Job properties to all Job creation methods.
• Add support for lists of URLs to #load and #load_job. (jeremywadsack)
• Fix Schema::Field type helpers.
• Fix Table#load example in README.

1.1.0 / 2018-02-27

• Support table partitioning by field.
• Support Shared Configuration.
• Improve AsyncInserter performance.

1.0.0 / 2018-01-10

• Release 1.0.0
• Update authentication documentation
  • Update Data documentation and code examples
  • Remove reference to sync and async queries
• Allow use of URI objects for Dataset#load, Table#load, and Table#load_job

0.30.0 / 2017-11-14

• Add Google::Cloud::Bigquery::Credentials class.
• Rename constructor arguments to project_id and credentials. (The previous arguments project and keyfile are still supported.)
• Support creating Dataset and Table objects without making API calls using skip_lookup argument.
  • Add Dataset#reference? and Dataset#resource? helper method.
  • Add Table#reference? and Table#resource? and Table#resource_partial? and Table#resource_full? helper methods.
• Dataset#insert_async and Dataset#insert_async now yields a Table::AsyncInserter::Result object.
• View is removed, now uses Table class.
  • Needed to support skip_lookup argument.
  • Calling Table#data on a view now raises (breaking change).
• Performance improvements for queries.
• Updated google-api-client, googleauth dependencies.

0.29.0 / 2017-10-09

This is a major release with many new features and several breaking changes.

Major Changes

• All queries now use a new implementation, using a job and polling for results.
• The copy, load, extract methods now all have high-level and low-level versions, similar to query and query_job.
• Added asynchronous row insertion, allowing data to be collected and inserted in batches.
• Support external data sources for both queries and table views.
• Added create-on-insert support for tables.
• Allow for customizing job IDs to aid in organizing jobs.

Change Details

• Update high-level queries as follows:
  • Update QueryJob#wait_until_done! to use getQueryResults.
  • Update Project#query and Dataset#query with breaking changes:
    • Remove timeout and dryrun parameters.
    • Change return type from QueryData to Data.
  • Add QueryJob#data
  • Alias QueryJob#query_results to QueryJob#data with breaking changes:
    • Remove the timeout parameter.
    • Change the return type from QueryData to Data.
  • Update View#data with breaking changes:
    • Remove the timeout and dryrun parameters.
    • Change the return type from QueryData to Data.
  • Remove QueryData.
  • Update Project#query and Dataset#query with improved errors, replacing the previous simple error with one that contains all available information for why the job failed.
• Rename Dataset#load to Dataset#load_job; add high-level, synchronous version as Dataset#load.
• Rename Table#copy to Table#copy_job; add high-level, synchronous version as Table#copy.
• Rename Table#extract to Table#extract_job; add high-level, synchronous version as Table#extract.
• Rename Table#load to Table#load_job; add high-level, synchronous version as Table#load.
• Add support for querying external data sources with External.
• Add Table::AsyncInserter, Dataset#insert_async and Table#insert_async to collect and insert rows in batches.
• Add Dataset#insert to support creating a table while inserting rows if the table does not exist.
• Update retry logic to conform to the BigQuery SLA.
  • Use a minimum back-off interval of 1 second; for each consecutive error, increase the back-off interval exponentially up to 32 seconds.
  • Retry if all error reasons are retriable, not if any of the error reasons are retriable.
• Add support for labels to Dataset, Table, View and Job.
  • Add filter option to Project#datasets and Project#jobs.
• Add support for user-defined functions to Project#query_job, Dataset#query_job, QueryJob and View.
• In Dataset, Table, and View updates, add the use of ETags for optimistic concurrency control.
• Update Dataset#load and Table#load:
  • Add null_marker option and LoadJob#null_marker.
  • Add autodetect option and LoadJob#autodetect?.
• Fix the default value for LoadJob#quoted_newlines?.
• Add job_id and prefix options for controlling client-side job ID generation to Project#query_job, Dataset#load, Dataset#query_job, Table#copy, Table#extract, and Table#load.
• Add Job#user_email.
• Set the maximum delay of Job#wait_until_done! polling to 60 seconds.
• Automatically retry Job#cancel.
• Allow users to specify if a View query is using Standard vs. Legacy SQL.
• Add project option to Project#query_job.
• Add QueryJob#query_plan, QueryJob::Stage and QueryJob::Step to expose query plan information.
• Add Table#buffer_bytes, Table#buffer_rows and Table#buffer_oldest_at to expose streaming buffer information.
• Update Dataset#insert and Table#insert to raise an error if rows is empty.
• Update Error with a mapping from code 412 to FailedPreconditionError.
• Update Data#schema to freeze the returned Schema object (as in View and LoadJob.)

0.28.0 / 2017-09-28

• Update Google API Client dependency to 0.14.x.

0.27.1 / 2017-07-11

• Add InsertResponse::InsertError#index (zedalaye)

0.27.0 / 2017-06-28

• Add maximum_billing_tier and maximum_bytes_billed to QueryJob, Project#query_job and Dataset#query_job.
• Add Dataset#load to support creating, configuring and loading a table in one API call.
• Add Project#schema.
• Upgrade dependency on Google API Client.
• Update gem spec homepage links.
• Update examples of field access to use symbols instead of strings in the documentation.

0.26.0 / 2017-04-05

• Upgrade dependency on Google API Client

0.25.0 / 2017-03-31

• Add #cancel to Job
• Updated documentation

0.24.0 / 2017-03-03

Major release, several new features, some breaking changes.

• Standard SQL is now the default syntax.
• Legacy SQL syntax can be enabled by providing legacy_sql: true.
• Several fixes to how data values are formatted when returned from BigQuery.
• Returned data rows are now hashes with Symbol keys instead of String keys.
• Several fixes to how data values are formatted when importing to BigQuery.
• Several improvements to manipulating table schema fields.
• Removal of Schema#fields= and Data#raw methods.
• Removal of fields argument from Dataset#create_table method.
• Dependency on Google API Client has been updated to 0.10.x.

0.23.0 / 2016-12-8

• Support Query Parameters using params method arguments to query and query_job
• Add standard_sql/legacy_sql method arguments to to query and query_job
• Add standard_sql?/legacy_sql? attributes to QueryJob
• Many documentation improvements

0.21.0 / 2016-10-20

• New service constructor Google::Cloud::Bigquery.new

0.20.2 / 2016-09-30

• Add list of projects that the current credentials can access. (remi)

0.20.1 / 2016-09-02

• Fix for timeout on uploads.

0.20.0 / 2016-08-26

This gem contains the Google BigQuery service implementation for the google-cloud gem. The google-cloud gem replaces the old gcloud gem. Legacy code can continue to use the gcloud gem.

• Namespace is now Google::Cloud
• The google-cloud gem is now an umbrella package for individual gems