[[["易于理解","easyToUnderstand","thumb-up"],["解决了我的问题","solvedMyProblem","thumb-up"],["其他","otherUp","thumb-up"]],[["很难理解","hardToUnderstand","thumb-down"],["信息或示例代码不正确","incorrectInformationOrSampleCode","thumb-down"],["没有我需要的信息/示例","missingTheInformationSamplesINeed","thumb-down"],["翻译问题","translationIssue","thumb-down"],["其他","otherDown","thumb-down"]],["最后更新时间 (UTC)：2025-08-18。"],[[["\u003cp\u003eGoogle provides open-source Dataflow templates that eliminate the need to write custom pipeline code for various data processing tasks.\u003c/p\u003e\n"],["\u003cp\u003eThese templates are categorized into Streaming, Batch, and Utility, catering to different data processing requirements like continuous data flow and bulk data operations.\u003c/p\u003e\n"],["\u003cp\u003eStreaming templates offer capabilities such as real-time data transfer from Kafka, Pub/Sub, and other sources to BigQuery and Cloud Storage.\u003c/p\u003e\n"],["\u003cp\u003eBatch templates enable bulk data transfer and transformations, supporting operations like moving data between BigQuery, Cloud Storage, Spanner, and various database systems.\u003c/p\u003e\n"],["\u003cp\u003eThe utility templates provide functionality for things like bulk file operations, and data generation to various sources.\u003c/p\u003e\n"]]],[],null,["# Google-provided templates\n\nGoogle provides\n[open source](https://github.com/GoogleCloudPlatform/DataflowTemplates)\nDataflow templates that you can use instead of writing pipeline code. This page\nlists the available templates. Container images for these templates are hosted at\n[`gcr.io/dataflow-templates`](https://gcr.io/dataflow-templates).\n\n- For general information about templates, see the [Overview](/dataflow/docs/guides/templates/overview).\n- To get started, run the sample template [WordCount](/dataflow/docs/sample-template).\n- To create your own template, see how to [extend templates](/dataflow/docs/guides/templates/overview#extend).\n\nStreaming templates\n-------------------\n\nTemplates for processing data continuously:\n\n- [Apache Kafka to Apache Kafka](/dataflow/docs/guides/templates/provided/kafka-to-kafka)\n- [Apache Kafka to BigQuery](/dataflow/docs/guides/templates/provided/kafka-to-bigquery)\n- [Apache Kafka to Cloud Storage](/dataflow/docs/guides/templates/provided/kafka-to-gcs)\n- [Change Data Capture from MySQL to BigQuery (Stream)](/dataflow/docs/guides/templates/provided/mysql-change-data-capture-to-bigquery)\n- [Bigtable change streams to BigQuery](/dataflow/docs/guides/templates/provided/cloud-bigtable-change-streams-to-bigquery)\n- [Bigtable change streams to Pub/Sub](/dataflow/docs/guides/templates/provided/cloud-bigtable-change-streams-to-pubsub)\n- [Spanner change streams to BigQuery](/dataflow/docs/guides/templates/provided/cloud-spanner-change-streams-to-bigquery)\n- [Spanner change streams to Cloud Storage](/dataflow/docs/guides/templates/provided/cloud-spanner-change-streams-to-cloud-storage)\n- [Spanner change streams to any source database](/dataflow/docs/guides/templates/provided/spanner-to-sourcedb)\n- [Spanner change streams to Pub/Sub](/dataflow/docs/guides/templates/provided/cloud-spanner-change-streams-to-pubsub)\n- [Datastream to BigQuery (Stream)](/dataflow/docs/guides/templates/provided/datastream-to-bigquery)\n- [Datastream to Spanner](/dataflow/docs/guides/templates/provided/datastream-to-cloud-spanner)\n- [Datastream to MySQL or PostgreSQL (Stream)](/dataflow/docs/guides/templates/provided/datastream-to-sql)\n- [Data Masking/Tokenization from Cloud Storage to BigQuery (using Cloud DLP)](/dataflow/docs/guides/templates/provided/dlp-text-to-bigquery)\n- [JMS to Pub/Sub](/dataflow/docs/guides/templates/provided/jms-to-pubsub)\n- [MongoDB to BigQuery (Stream)](/dataflow/docs/guides/templates/provided/mongodb-change-stream-to-bigquery)\n- [MQTT to Pub/Sub](/dataflow/docs/guides/templates/provided/mqtt-to-pubsub)\n- [Pub/Sub to BigQuery](/dataflow/docs/guides/templates/provided/pubsub-to-bigquery)\n- [Pub/Sub to BigQuery with Python UDF](/dataflow/docs/guides/templates/provided/pubsub-to-bigquery-python-udf)\n- [Pub/Sub Avro to BigQuery](/dataflow/docs/guides/templates/provided/pubsub-avro-to-bigquery)\n- [Pub/Sub Proto to BigQuery](/dataflow/docs/guides/templates/provided/pubsub-proto-to-bigquery)\n- [Pub/Sub Proto to BigQuery with Python UDF](/dataflow/docs/guides/templates/provided/pubsub-proto-to-bigquery-python-udf)\n- [Pub/Sub Subscription to BigQuery](/dataflow/docs/guides/templates/provided/pubsub-subscription-to-bigquery)\n- [Pub/Sub to Avro Files on Cloud Storage](/dataflow/docs/guides/templates/provided/pubsub-to-avro)\n- [Pub/Sub to Datadog](/dataflow/docs/guides/templates/provided/pubsub-to-datadog)\n- [Pub/Sub to Elasticsearch](/dataflow/docs/guides/templates/provided/pubsub-to-elasticsearch)\n- [Pub/Sub to Java Database Connectivity (JDBC)](/dataflow/docs/guides/templates/provided/pubsub-to-jdbc)\n- [Pub/Sub to MongoDB](/dataflow/docs/guides/templates/provided/pubsub-to-mongodb)\n- [Pub/Sub to MongoDB with Python UDF](/dataflow/docs/guides/templates/provided/pubsub-to-mongodb-python-udf)\n- [Pub/Sub to Pub/Sub](/dataflow/docs/guides/templates/provided/pubsub-to-pubsub)\n- [Pub/Sub to Redis](/dataflow/docs/guides/templates/provided/pubsub-to-redis)\n- [Pub/Sub to Splunk](/dataflow/docs/guides/templates/provided/pubsub-to-splunk)\n- [Pub/Sub Topic or Subscription to Text Files on Cloud Storage](/dataflow/docs/guides/templates/provided/pubsub-topic-subscription-to-text)\n- [Pub/Sub Topic to Text Files on Cloud Storage](/dataflow/docs/guides/templates/provided/pubsub-topic-to-text)\n- [Text Files on Cloud Storage to BigQuery (Stream)](/dataflow/docs/guides/templates/provided/text-to-bigquery-stream)\n- [Cloud Storage text files to BigQuery with Python UDF](/dataflow/docs/guides/templates/provided/text-to-bigquery-stream-python-udf)\n- [Text files on Cloud Storage to Pub/Sub (Stream)](/dataflow/docs/guides/templates/provided/text-to-pubsub-stream)\n\nBatch templates\n---------------\n\nTemplates for processing data in bulk:\n\n- [Any Source DB to Spanner](/dataflow/docs/guides/templates/provided/sourcedb-to-spanner)\n- [Apache Cassandra to Bigtable](/dataflow/docs/guides/templates/provided/cassandra-to-bigtable)\n- [AstraDB to BigQuery](/dataflow/docs/guides/templates/provided/astradb-to-bigquery)\n- [BigQuery to Bigtable](/dataflow/docs/guides/templates/provided/bigquery-to-bigtable)\n- [BigQuery to Clickhouse](/dataflow/docs/guides/templates/provided/bigquery-to-clickhouse)\n- [BigQuery export to Parquet (via Storage API)](/dataflow/docs/guides/templates/provided/bigquery-to-parquet)\n- [BigQuery to Cloud Storage TFRecords](/dataflow/docs/guides/templates/provided/bigquery-to-tfrecords)\n- [BigQuery to Elasticsearch](/dataflow/docs/guides/templates/provided/bigquery-to-elasticsearch)\n- [BigQuery to MongoDB](/dataflow/docs/guides/templates/provided/bigquery-to-mongodb)\n- [Bigtable to Cloud Storage Avro](/dataflow/docs/guides/templates/provided/bigtable-to-avro)\n- [Bigtable to Cloud Storage JSON](/dataflow/docs/guides/templates/provided/bigtable-to-json)\n- [Bigtable to Cloud Storage Parquet](/dataflow/docs/guides/templates/provided/bigtable-to-parquet)\n- [Bigtable to Cloud Storage SequenceFile](/dataflow/docs/guides/templates/provided/bigtable-to-sequencefile)\n- [Bigtable change streams to Vertex AI Vector Search](/dataflow/docs/guides/templates/provided/bigtable-change-streams-to-vector-search)\n- [Bigtable to Vertex AI Vector Search files on Cloud Storage](/dataflow/docs/guides/templates/provided/bigtable-to-vector-embeddings)\n- [Spanner to BigQuery](/dataflow/docs/guides/templates/provided/cloud-spanner-to-bigquery)\n- [Spanner to Cloud Storage Avro](/dataflow/docs/guides/templates/provided/cloud-spanner-to-avro)\n- [Spanner to Cloud Storage Text](/dataflow/docs/guides/templates/provided/cloud-spanner-to-cloud-storage)\n- [Spanner to Vertex AI Vector Search files on Cloud Storage](/dataflow/docs/guides/templates/provided/cloud-spanner-to-vertex-vector-search)\n- [Cloud Storage Avro to Bigtable](/dataflow/docs/guides/templates/provided/avro-to-bigtable)\n- [Cloud Storage Parquet to Bigtable](/dataflow/docs/guides/templates/provided/parquet-to-bigtable)\n- [Cloud Storage SequenceFile to Bigtable](/dataflow/docs/guides/templates/provided/sequencefile-to-bigtable)\n- [Cloud Storage Avro to Spanner](/dataflow/docs/guides/templates/provided/avro-to-cloud-spanner)\n- [Cloud Storage CSV files to BigQuery](/dataflow/docs/guides/templates/provided/cloud-storage-csv-to-bigquery)\n- [Cloud Storage Text to BigQuery](/dataflow/docs/guides/templates/provided/cloud-storage-to-bigquery)\n- [Cloud Storage text files to BigQuery with Python UDF](/dataflow/docs/guides/templates/provided/cloud-storage-to-bigquery-python-udf)\n- [Cloud Storage Text to Spanner](/dataflow/docs/guides/templates/provided/cloud-storage-to-cloud-spanner)\n- [Cloud Storage Text to Datastore](/dataflow/docs/guides/templates/provided/cloud-storage-to-datastore) \\[Deprecated\\]\n- [Cloud Storage Text to Firestore (Datastore mode)](/dataflow/docs/guides/templates/provided/cloud-storage-to-firestore)\n- [Cloud Storage Text to Pub/Sub (Batch)](/dataflow/docs/guides/templates/provided/cloud-storage-to-pubsub)\n- [Cloud Storage to Elasticsearch](/dataflow/docs/guides/templates/provided/cloud-storage-to-elasticsearch)\n- [Datastore to Cloud Storage Text](/dataflow/docs/guides/templates/provided/datastore-to-cloud-storage) \\[Deprecated\\]\n- [Firestore (Datastore mode) to Cloud Storage Text](/dataflow/docs/guides/templates/provided/firestore-to-cloud-storage)\n- [Google Ads to BigQuery](/dataflow/docs/guides/templates/provided/google-ads-to-bigquery)\n- [Google Cloud to Neo4j](/dataflow/docs/guides/templates/provided/google-cloud-to-neo4j)\n- [JDBC to BigQuery](/dataflow/docs/guides/templates/provided/jdbc-to-bigquery)\n- [JDBC to Pub/Sub](/dataflow/docs/guides/templates/provided/jdbc-to-pubsub)\n- [MongoDB to BigQuery](/dataflow/docs/guides/templates/provided/mongodb-to-bigquery)\n- [MySQL to BigQuery](/dataflow/docs/guides/templates/provided/mysql-to-bigquery)\n- [Oracle to BigQuery](/dataflow/docs/guides/templates/provided/oracle-to-bigquery)\n- [PostgreSQL to BigQuery](/dataflow/docs/guides/templates/provided/postgresql-to-bigquery)\n- [SQL Server to BigQuery](/dataflow/docs/guides/templates/provided/sqlserver-to-bigquery)\n\nUtility templates\n-----------------\n\n- [Bulk Compress Cloud Storage Files](/dataflow/docs/guides/templates/provided/bulk-compress-cloud-storage)\n- [Bulk Decompress Cloud Storage Files](/dataflow/docs/guides/templates/provided/bulk-decompress-cloud-storage)\n- [Datastore Bulk Delete](/dataflow/docs/guides/templates/provided/datastore-bulk-delete) \\[Deprecated\\]\n- [File Format Conversion](/dataflow/docs/guides/templates/provided/file-format-conversion)\n- [Firestore (Datastore mode) Bulk Delete](/dataflow/docs/guides/templates/provided/firestore-bulk-delete)\n- [Streaming\n Data Generator to Pub/Sub, BigQuery, and Cloud Storage](/dataflow/docs/guides/templates/provided/streaming-data-generator)"]]
