Mit Sammlungen den Überblick behalten
Sie können Inhalte basierend auf Ihren Einstellungen speichern und kategorisieren.
Google bietet Open-Source-Vorlagen für Dataflow, die Sie verwenden können, anstatt Pipelinecode zu schreiben. Auf dieser Seite sind die verfügbaren Vorlagen aufgeführt. Container-Images für diese Vorlagen werden unter gcr.io/dataflow-templates gehostet.
Allgemeine Informationen zu Vorlagen finden Sie unter Übersicht.
Führen Sie zum Einstieg die Beispielvorlage WordCount aus.
Wie Sie eine eigene Vorlage erstellen, erfahren Sie unter Vorlagen erweitern.
Streamingvorlagen
Vorlagen für die kontinuierliche Verarbeitung von Daten:
[[["Leicht verständlich","easyToUnderstand","thumb-up"],["Mein Problem wurde gelöst","solvedMyProblem","thumb-up"],["Sonstiges","otherUp","thumb-up"]],[["Schwer verständlich","hardToUnderstand","thumb-down"],["Informationen oder Beispielcode falsch","incorrectInformationOrSampleCode","thumb-down"],["Benötigte Informationen/Beispiele nicht gefunden","missingTheInformationSamplesINeed","thumb-down"],["Problem mit der Übersetzung","translationIssue","thumb-down"],["Sonstiges","otherDown","thumb-down"]],["Zuletzt aktualisiert: 2025-08-18 (UTC)."],[[["\u003cp\u003eGoogle provides open-source Dataflow templates that eliminate the need to write custom pipeline code for various data processing tasks.\u003c/p\u003e\n"],["\u003cp\u003eThese templates are categorized into Streaming, Batch, and Utility, catering to different data processing requirements like continuous data flow and bulk data operations.\u003c/p\u003e\n"],["\u003cp\u003eStreaming templates offer capabilities such as real-time data transfer from Kafka, Pub/Sub, and other sources to BigQuery and Cloud Storage.\u003c/p\u003e\n"],["\u003cp\u003eBatch templates enable bulk data transfer and transformations, supporting operations like moving data between BigQuery, Cloud Storage, Spanner, and various database systems.\u003c/p\u003e\n"],["\u003cp\u003eThe utility templates provide functionality for things like bulk file operations, and data generation to various sources.\u003c/p\u003e\n"]]],[],null,["# Google-provided templates\n\nGoogle provides\n[open source](https://github.com/GoogleCloudPlatform/DataflowTemplates)\nDataflow templates that you can use instead of writing pipeline code. This page\nlists the available templates. Container images for these templates are hosted at\n[`gcr.io/dataflow-templates`](https://gcr.io/dataflow-templates).\n\n- For general information about templates, see the [Overview](/dataflow/docs/guides/templates/overview).\n- To get started, run the sample template [WordCount](/dataflow/docs/sample-template).\n- To create your own template, see how to [extend templates](/dataflow/docs/guides/templates/overview#extend).\n\nStreaming templates\n-------------------\n\nTemplates for processing data continuously:\n\n- [Apache Kafka to Apache Kafka](/dataflow/docs/guides/templates/provided/kafka-to-kafka)\n- [Apache Kafka to BigQuery](/dataflow/docs/guides/templates/provided/kafka-to-bigquery)\n- [Apache Kafka to Cloud Storage](/dataflow/docs/guides/templates/provided/kafka-to-gcs)\n- [Change Data Capture from MySQL to BigQuery (Stream)](/dataflow/docs/guides/templates/provided/mysql-change-data-capture-to-bigquery)\n- [Bigtable change streams to BigQuery](/dataflow/docs/guides/templates/provided/cloud-bigtable-change-streams-to-bigquery)\n- [Bigtable change streams to Pub/Sub](/dataflow/docs/guides/templates/provided/cloud-bigtable-change-streams-to-pubsub)\n- [Spanner change streams to BigQuery](/dataflow/docs/guides/templates/provided/cloud-spanner-change-streams-to-bigquery)\n- [Spanner change streams to Cloud Storage](/dataflow/docs/guides/templates/provided/cloud-spanner-change-streams-to-cloud-storage)\n- [Spanner change streams to any source database](/dataflow/docs/guides/templates/provided/spanner-to-sourcedb)\n- [Spanner change streams to Pub/Sub](/dataflow/docs/guides/templates/provided/cloud-spanner-change-streams-to-pubsub)\n- [Datastream to BigQuery (Stream)](/dataflow/docs/guides/templates/provided/datastream-to-bigquery)\n- [Datastream to Spanner](/dataflow/docs/guides/templates/provided/datastream-to-cloud-spanner)\n- [Datastream to MySQL or PostgreSQL (Stream)](/dataflow/docs/guides/templates/provided/datastream-to-sql)\n- [Data Masking/Tokenization from Cloud Storage to BigQuery (using Cloud DLP)](/dataflow/docs/guides/templates/provided/dlp-text-to-bigquery)\n- [JMS to Pub/Sub](/dataflow/docs/guides/templates/provided/jms-to-pubsub)\n- [MongoDB to BigQuery (Stream)](/dataflow/docs/guides/templates/provided/mongodb-change-stream-to-bigquery)\n- [MQTT to Pub/Sub](/dataflow/docs/guides/templates/provided/mqtt-to-pubsub)\n- [Pub/Sub to BigQuery](/dataflow/docs/guides/templates/provided/pubsub-to-bigquery)\n- [Pub/Sub to BigQuery with Python UDF](/dataflow/docs/guides/templates/provided/pubsub-to-bigquery-python-udf)\n- [Pub/Sub Avro to BigQuery](/dataflow/docs/guides/templates/provided/pubsub-avro-to-bigquery)\n- [Pub/Sub Proto to BigQuery](/dataflow/docs/guides/templates/provided/pubsub-proto-to-bigquery)\n- [Pub/Sub Proto to BigQuery with Python UDF](/dataflow/docs/guides/templates/provided/pubsub-proto-to-bigquery-python-udf)\n- [Pub/Sub Subscription to BigQuery](/dataflow/docs/guides/templates/provided/pubsub-subscription-to-bigquery)\n- [Pub/Sub to Avro Files on Cloud Storage](/dataflow/docs/guides/templates/provided/pubsub-to-avro)\n- [Pub/Sub to Datadog](/dataflow/docs/guides/templates/provided/pubsub-to-datadog)\n- [Pub/Sub to Elasticsearch](/dataflow/docs/guides/templates/provided/pubsub-to-elasticsearch)\n- [Pub/Sub to Java Database Connectivity (JDBC)](/dataflow/docs/guides/templates/provided/pubsub-to-jdbc)\n- [Pub/Sub to MongoDB](/dataflow/docs/guides/templates/provided/pubsub-to-mongodb)\n- [Pub/Sub to MongoDB with Python UDF](/dataflow/docs/guides/templates/provided/pubsub-to-mongodb-python-udf)\n- [Pub/Sub to Pub/Sub](/dataflow/docs/guides/templates/provided/pubsub-to-pubsub)\n- [Pub/Sub to Redis](/dataflow/docs/guides/templates/provided/pubsub-to-redis)\n- [Pub/Sub to Splunk](/dataflow/docs/guides/templates/provided/pubsub-to-splunk)\n- [Pub/Sub Topic or Subscription to Text Files on Cloud Storage](/dataflow/docs/guides/templates/provided/pubsub-topic-subscription-to-text)\n- [Pub/Sub Topic to Text Files on Cloud Storage](/dataflow/docs/guides/templates/provided/pubsub-topic-to-text)\n- [Text Files on Cloud Storage to BigQuery (Stream)](/dataflow/docs/guides/templates/provided/text-to-bigquery-stream)\n- [Cloud Storage text files to BigQuery with Python UDF](/dataflow/docs/guides/templates/provided/text-to-bigquery-stream-python-udf)\n- [Text files on Cloud Storage to Pub/Sub (Stream)](/dataflow/docs/guides/templates/provided/text-to-pubsub-stream)\n\nBatch templates\n---------------\n\nTemplates for processing data in bulk:\n\n- [Any Source DB to Spanner](/dataflow/docs/guides/templates/provided/sourcedb-to-spanner)\n- [Apache Cassandra to Bigtable](/dataflow/docs/guides/templates/provided/cassandra-to-bigtable)\n- [AstraDB to BigQuery](/dataflow/docs/guides/templates/provided/astradb-to-bigquery)\n- [BigQuery to Bigtable](/dataflow/docs/guides/templates/provided/bigquery-to-bigtable)\n- [BigQuery to Clickhouse](/dataflow/docs/guides/templates/provided/bigquery-to-clickhouse)\n- [BigQuery export to Parquet (via Storage API)](/dataflow/docs/guides/templates/provided/bigquery-to-parquet)\n- [BigQuery to Cloud Storage TFRecords](/dataflow/docs/guides/templates/provided/bigquery-to-tfrecords)\n- [BigQuery to Elasticsearch](/dataflow/docs/guides/templates/provided/bigquery-to-elasticsearch)\n- [BigQuery to MongoDB](/dataflow/docs/guides/templates/provided/bigquery-to-mongodb)\n- [Bigtable to Cloud Storage Avro](/dataflow/docs/guides/templates/provided/bigtable-to-avro)\n- [Bigtable to Cloud Storage JSON](/dataflow/docs/guides/templates/provided/bigtable-to-json)\n- [Bigtable to Cloud Storage Parquet](/dataflow/docs/guides/templates/provided/bigtable-to-parquet)\n- [Bigtable to Cloud Storage SequenceFile](/dataflow/docs/guides/templates/provided/bigtable-to-sequencefile)\n- [Bigtable change streams to Vertex AI Vector Search](/dataflow/docs/guides/templates/provided/bigtable-change-streams-to-vector-search)\n- [Bigtable to Vertex AI Vector Search files on Cloud Storage](/dataflow/docs/guides/templates/provided/bigtable-to-vector-embeddings)\n- [Spanner to BigQuery](/dataflow/docs/guides/templates/provided/cloud-spanner-to-bigquery)\n- [Spanner to Cloud Storage Avro](/dataflow/docs/guides/templates/provided/cloud-spanner-to-avro)\n- [Spanner to Cloud Storage Text](/dataflow/docs/guides/templates/provided/cloud-spanner-to-cloud-storage)\n- [Spanner to Vertex AI Vector Search files on Cloud Storage](/dataflow/docs/guides/templates/provided/cloud-spanner-to-vertex-vector-search)\n- [Cloud Storage Avro to Bigtable](/dataflow/docs/guides/templates/provided/avro-to-bigtable)\n- [Cloud Storage Parquet to Bigtable](/dataflow/docs/guides/templates/provided/parquet-to-bigtable)\n- [Cloud Storage SequenceFile to Bigtable](/dataflow/docs/guides/templates/provided/sequencefile-to-bigtable)\n- [Cloud Storage Avro to Spanner](/dataflow/docs/guides/templates/provided/avro-to-cloud-spanner)\n- [Cloud Storage CSV files to BigQuery](/dataflow/docs/guides/templates/provided/cloud-storage-csv-to-bigquery)\n- [Cloud Storage Text to BigQuery](/dataflow/docs/guides/templates/provided/cloud-storage-to-bigquery)\n- [Cloud Storage text files to BigQuery with Python UDF](/dataflow/docs/guides/templates/provided/cloud-storage-to-bigquery-python-udf)\n- [Cloud Storage Text to Spanner](/dataflow/docs/guides/templates/provided/cloud-storage-to-cloud-spanner)\n- [Cloud Storage Text to Datastore](/dataflow/docs/guides/templates/provided/cloud-storage-to-datastore) \\[Deprecated\\]\n- [Cloud Storage Text to Firestore (Datastore mode)](/dataflow/docs/guides/templates/provided/cloud-storage-to-firestore)\n- [Cloud Storage Text to Pub/Sub (Batch)](/dataflow/docs/guides/templates/provided/cloud-storage-to-pubsub)\n- [Cloud Storage to Elasticsearch](/dataflow/docs/guides/templates/provided/cloud-storage-to-elasticsearch)\n- [Datastore to Cloud Storage Text](/dataflow/docs/guides/templates/provided/datastore-to-cloud-storage) \\[Deprecated\\]\n- [Firestore (Datastore mode) to Cloud Storage Text](/dataflow/docs/guides/templates/provided/firestore-to-cloud-storage)\n- [Google Ads to BigQuery](/dataflow/docs/guides/templates/provided/google-ads-to-bigquery)\n- [Google Cloud to Neo4j](/dataflow/docs/guides/templates/provided/google-cloud-to-neo4j)\n- [JDBC to BigQuery](/dataflow/docs/guides/templates/provided/jdbc-to-bigquery)\n- [JDBC to Pub/Sub](/dataflow/docs/guides/templates/provided/jdbc-to-pubsub)\n- [MongoDB to BigQuery](/dataflow/docs/guides/templates/provided/mongodb-to-bigquery)\n- [MySQL to BigQuery](/dataflow/docs/guides/templates/provided/mysql-to-bigquery)\n- [Oracle to BigQuery](/dataflow/docs/guides/templates/provided/oracle-to-bigquery)\n- [PostgreSQL to BigQuery](/dataflow/docs/guides/templates/provided/postgresql-to-bigquery)\n- [SQL Server to BigQuery](/dataflow/docs/guides/templates/provided/sqlserver-to-bigquery)\n\nUtility templates\n-----------------\n\n- [Bulk Compress Cloud Storage Files](/dataflow/docs/guides/templates/provided/bulk-compress-cloud-storage)\n- [Bulk Decompress Cloud Storage Files](/dataflow/docs/guides/templates/provided/bulk-decompress-cloud-storage)\n- [Datastore Bulk Delete](/dataflow/docs/guides/templates/provided/datastore-bulk-delete) \\[Deprecated\\]\n- [File Format Conversion](/dataflow/docs/guides/templates/provided/file-format-conversion)\n- [Firestore (Datastore mode) Bulk Delete](/dataflow/docs/guides/templates/provided/firestore-bulk-delete)\n- [Streaming\n Data Generator to Pub/Sub, BigQuery, and Cloud Storage](/dataflow/docs/guides/templates/provided/streaming-data-generator)"]]
