Stay organized with collections
Save and categorize content based on your preferences.
Dataflow has ended support for Dataflow SQL.
This is a breaking change.
As of July 31, 2024, you can't access Dataflow SQL in the
Google Cloud console.
As of January 31, 2025, you can't use Dataflow SQL in the
Google Cloud CLI.
Replacement: Beam SQL
Beam SQL lets you
query bounded and unbounded PCollections with SQL statements. If you're
working in an Apache Beam notebook, you can use the IPython
custom magicbeam_sql to speed up your pipeline development.
Beam SQL provides similar functionality as Dataflow SQL,
using SQL statements to query PCollections.
Migration
We recommend that you begin migrating your Dataflow SQL query to Beam SQL
as soon as possible.
Moving to Beam SQL is a breaking change that
affects any Dataflow SQL job or query. You will need to rebuild these
jobs and queries.
With this change, you can't create new Dataflow SQL jobs. However, this
change doesn't affect Dataflow SQL jobs that are already running.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-04-12 UTC."],[[["Dataflow SQL is losing support, effective July 31, 2024, in the Google Cloud console and January 31, 2025, in the Google Cloud CLI."],["Beam SQL is the recommended replacement for Dataflow SQL, offering similar functionality for querying `PCollections` with SQL."],["Migrating from Dataflow SQL to Beam SQL requires rebuilding existing jobs and queries, as it is a breaking change."],["New Dataflow SQL jobs cannot be created, but this change does not impact previously existing, currently running Dataflow SQL jobs."]]],[]]
