You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.
To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly.
December 19, 2024
You can now manage Dataform repositories in Dataplex. Metadata of Dataform repositories is automatically available in Dataplex, without additional configuration. For more information, see Manage Dataform assets with Dataplex. This feature is generally available (GA).
You can now search for and view the metadata of Dataform repositories in the Dataplex console. This feature is in preview.
October 10, 2024
Dataform is available in the following regions:
- asia-northeast2
- asia-south2
- australia-southeast2
- europe-central2
- europe-north1
- europe-west8
- europe-west9
- europe-west10
- me-west1
- northamerica-northeast2
- southamerica-west1
- us-west3
For more information, see Locations.
September 30, 2024
The maximum size limit for workspaces encrypted with customer-managed encryption keys (CMEK) is 512 MB. For more information about Dataform quotas and limits, see Quotas and limits. For more information about encrypting Dataform repositories with CMEK, see Use customer-managed encryption keys.
September 09, 2024
You can now set a default Dataform customer-managed encryption keys (CMEK) key for your project to encrypt multiple Dataform repositories with the same CMEK key. For more information, see Use Dataform default CMEK keys.
August 12, 2024
Customer-Managed Encryption Keys (CMEK) in Dataform are generally available (GA). For more information, see Use customer-managed encryption keys.
You can now use Cloud External Key Manager (Cloud EKM) keys to protect Dataform data. Cloud EKM keys in Dataform are generally available (GA). For more information, see Using and managing external keys.
August 05, 2024
Access Approval supports Dataform in the GA stage.
July 31, 2024
The maximum size limit for repositories encrypted with customer-managed encryption keys (CMEK) is 512 MB. For more information about Dataform quotas and limits, see Quotas and limits. For more information about encrypting Dataform repositories with CMEK, see Use customer-managed encryption keys.
July 18, 2024
As of Dataform Core 3.0.0.,
Dataform doesn't distribute a Docker image. You can build your own
Docker image of Dataform, which you can use to run the equivalent of Dataform CLI commands. To build your own Docker image, see Containerize an application
in the Docker documentation.
June 21, 2024
The 3.0.0 version of the open-source Dataform framework is available.
The workflow_settings.yaml file, which was introduced in Dataform Core 3.0.0-beta.0, replaces dataform.json.
You can specify the Dataform Core version directly in the workflow_settings.yaml file, which removes the need for package.json for most repositories. To have package dependencies other than @dataform/core, the package.json file is still required.
No immediate action to convert existing Dataform code is required. You can continue to use dataform.json and package.json in existing repositories.
You can convert your dataform.json file into workflow_settings.yaml by following the instructions in the 3.0.0 GitHub release.
New repositories use workflow_settings.yaml by default. You can replace the workflow_settings.yaml file with dataform.json to continue using the JSON format. If you remove workflow_settings.yaml, you need to add a package.json file to your repository to install @dataform/core.
For more information, see the 3.0.0 release on GitHub.
June 17, 2024
You can now inspect past manual compilation results of a selected release configuration. For more information, see View details of a release configuration.
May 29, 2024
Dataform Core includeDependentAssertions and dependOnDependencyAssertions parameters for adding assertions as dependencies are available.
You can set the includeDependentAssertions parameter in a selected action to automatically add assertions of a selected dependency action as dependencies of the edited action.
You can set dependOnDependencyAssertions the parameter in a selected action to automatically add assertions of all dependency actions as dependencies of the edited action.
For more information, see Set assertions as dependencies.
May 10, 2024
Gemini, an AI-powered collaborator in Google Cloud, can help you generate code in Dataform. This feature is in preview. For more information, see Write queries with Gemini assistance.
April 02, 2024
You can now use Customer-Managed Encryption Keys (CMEK) to protect repositories in Dataform. CMEK in Dataform is available in preview. For more information, see Use customer-managed encryption keys.
March 13, 2024
The 3.0.0-beta.0 version of the open-source Dataform framework is available. This update introduces significant changes, including, but not limited to, the following:
- Deprecation of
dataform.jsonin favor ofworkflow_settings.yaml - Stateless package installation by
@dataform/cli - Warehouse-agnostic compilation output
You don't need to take immediate action to update your Dataform code.
For more information, see the 3.0.0-beta.0 release on GitHub.
March 12, 2024
Granting repository access to all authenticated users is available. For more information, see Grant public access to a repository.
March 11, 2024
Dataform is available in the africa-south1 region. For more information, see Locations.
February 26, 2024
Dataform is available in the us-south1 region. For more information, see Locations.
February 21, 2024
Support for VPC Service Controls is generally available (GA).
February 15, 2024
Dataform is available in the following regions:
- asia-east2
- asia-northeast3
- asia-southeast2
- europe-southwest1
- europe-west12
- me-central1
- me-central2
- northamerica-northeast
- us-east4
- us-east5
- us-west2
- us-west4
For more information, see Locations.
January 29, 2024
Support for VPC Service Controls is available in preview.
January 17, 2024
The Releases & Scheduling tab replaces the Release configurations and Workflow configurations tabs in repositories. Configuration and monitoring of release configurations and workflow configurations is available in the Releases & Scheduling tab.
For more information, see Create a release configuration, and Schedule executions with workflow configurations.
December 11, 2023
Display of files as workspace tabs is available.
December 07, 2023
The 2.8.0 version of the open-source Dataform framework is available. This update introduces performance improvements. For more information, see the 2.8.0: Packaging and compilation performance improvements release on GitHub.
December 05, 2023
December 04, 2023
Preview of uncommitted changes during a commit is available.
November 16, 2023
Dataform is compliant with VPAT.
For more information, see Dataform compliance.
October 27, 2023
Batch workspace deletion is available.
October 26, 2023
File search is available in workspaces.
October 23, 2023
The 2.7.0 version of the open-source Dataform framework is available. This update introduces explicitly listed column names in incremental insert statements. For more information, see the 2.7.0: Updates for Dataform GCP incremental SQL release on GitHub.
October 20, 2023
Workforce identity federation is available.
October 17, 2023
ssh_authentication_config and service_account fields are available in the google_dataform_repository Dataform Terraform resource.
October 13, 2023
Formatting of Dataform core and JavaScript code is available.
September 07, 2023
Service account delegation for workflow execution is now available. You can configure service account delegation for whole repositories, or for individual workflow configurations.
September 06, 2023
August 31, 2023
Legacy Dataform will be deprecated on February 26, 2024, after which you will not be able to access legacy projects.
We recommend migration to Dataform in Google Cloud Platform.
To learn more about Dataform in Google Cloud Platform, see Overview of Dataform.
To discover how to migrate legacy projects, see Migrate from legacy Dataform.
August 22, 2023
Dataform is compliant with SOC 2 and SOC 3.
For more information, see Dataform compliance.
August 18, 2023
SSH authentication for GitHub, GitLab, and Bitbucket Cloud repositories is supported.
August 09, 2023
Dataform is compliant with HIPAA, ISO/IEC 27001, and SOC 1.
For more information, see Dataform compliance.
August 02, 2023
Filtering and sorting repositories are available.
August 01, 2023
The following Dataform API methods for version control are available:
CommitRepositoryChanges()ReadRepositoryFile()QueryRepositoryDirectoryContents()FetchRepositoryHistory()
The following fields of Repository are available:
labelsset_authenticated_user_admin
July 27, 2023
Selection of files for a commit is available.
June 19, 2023
Filtering the compiled graph in a workspace is available.
May 04, 2023
Dataform release configurations are available. Release configurations let you configure execution environments, for example, staging and production.
Dataform workflow configurations are available. Workflow configurations let you execute SQL workflows on a schedule.
April 18, 2023
Cloud Logging is available for Dataform in Preview.
March 24, 2023
Workspace compilation overrides are available in Preview.
March 21, 2023
Dataform in Preview is available in the following regions:
- australia-southeast1
- southamerica-east1
March 10, 2023
Dataform in Preview is available in the following regions:
- asia-south1
- europe-west6
March 09, 2023
Query preview in a workspace is available in Preview.
March 06, 2023
Dataform in Preview is available in the following regions:
- asia-northeast1
- europe-west2
- europe-west3
- us-east1
February 14, 2023
Dataform in Preview is available in the following regions:
- asia-southeast1
- europe-west1
- us-west1
January 27, 2023
Dataform in Preview is available in the asia-east1 region.
January 25, 2023
Workspace navigation tabs are available in Preview.
January 09, 2023
Display of a SQL workflow as a compiled graph in a workspace is available in Preview.
November 25, 2022
Real-time validation of compiled queries against BigQuery is available in Preview.
Definition of BigQuery variables in the pre_operations block of type: incremental queries that contain a uniqueKey is available in Preview.
October 21, 2022
Configuration of policy tags in the config block is available in Preview.
August 25, 2022
Dataform is available in Preview.