Google Cloud Managed Service for Apache Kafka pricing

Preview notice

Managed Service for Apache Kafka is not free to use during the Preview.

Pricing

The cost of operating an Managed Service for Apache Kafka consists of charges related to running Kafka clusters and Private Service Connect (PSC) fees for access to the cluster.

Cluster charges

Cluster charges will be billed using three SKUs in the region where you run the cluster:

• Compute, covering vCPU and RAM
• Storage, including local storage of data on brokers (Persistent Disk) and long term storage.
• Networking, covering charges for inter-zone data transfer for replication of data within a cluster.

Understanding compute costs

The vCPU and RAM usage is reported in terms of abstract units called slots. A simple way to think about this unit is that a single vCPU with 4GB of RAM corresponds to 1.5 slots. This means for example, that a 6 vCPU cluster with 4GiB of RAM per vCPU would cost 6 x (1 vCPU + 4GiB) * 1.5 slots * 6 cents / hour / slot = 54 cents / hour in us-central1.

You can, however, use different amounts of RAM per vCPU in your clusters. The conversion factors from compute resources to slots are

This means that when you use 1 GiB of RAM for an hour, you pay for 0.125 slots for that hour.

Note that an Managed Service for Apache Kafka slot is a unit analogous to but distinct from BigQuery slots. The two have the same price to simplify comparison.

Understanding storage costs

The local storage is provisioned automatically. As a rough guide, you should expect to pay for 100GB of local storage for every vCPU you provision.

The cost of persistent storage is determined by how much data you retain in your topics. You do not pay for provisioning persistent storage capacity. You are billed for only a single replica of data in persistent storage. This is independent of the cluster configuration. For example, if you publish 1GiB of data and retain it for 1 day, you will pay for 1GiB-day of persistent storage.

In contrast, local storage is used by every replica of the data. So if you publish 1 GiB of data you will typically use 3 GiB of local disk space. Persistent storage charges are based on the total volume of data stored, while local storage charges are based on the volume of provisioned storage. Therefore, the actual volume of data stored in local storage does not affect what you pay for local storage.

Understanding inter-zone network costs

A Managed Service for Apache Kafka cluster replicates data across multiple zones. You are charged for inter-zone data transfer within the cluster. The cost is $0.01 per 1 gibibyte. By default, the replication factor for a cluster is 3. This means you will pay for inter-zone data transfer for replicating data to 2 out of the 3 zones.

Let's say you have a cluster with a replication factor of 3, which is the default. You write 10 GiB of data within a single zone. With a replication factor of 3, your 10 GiB of data is copied to 2 other zones. A total of 20 GiB of data (10 GiB x 2 copies) is transferred. The cost for this inter-zone data transfer is $0.20 (20 GiB x $0.01/GiB).

Private Service Connect charges

Managed Service for Apache Kafka service sets up all Private Service Connect (PSC) endpoints for your service. These endpoints are resources in a project you own and are billed separately, as PSC charges. You can read more about how PSC is used here. The PSC charges include the following components:

• Hourly charge for each endpoint. A cluster has a minimum of four endpoints. Each endpoint costs $0.01 per hour. Since a cluster requires at least four endpoints, expect a minimum charge of $0.04 per hour for total endpoints for each cluster.
• VPC Consumer data processing charges. The cost for data transfer ranges from $0.004 to $0.01 per gigabyte (GiB). The exact price depends on various factors, so it's best to check the VPC pricing guide for the most current rates.

For the most up-to-date information about VPC pricing, see the subsection on "Using a Private Service Connect endpoint (forwarding rule) to access a published service" in the "Private Service Connect" section of the VPC pricing guide.

Total cost of ownership

The total cost of running an Managed Service for Apache Kafka cluster depends on the size of the cluster, the volume of data you publish and consume, as well as the duration for which you store the data. In addition, an important parameter to consider is the average utilization of your cluster. The utilization determines the balance of inter-zone data transfer charges and compute charges. As a rough guide, you can assume that a single vCPU can handle 20MiBps of publish traffic and 80MiBps of consumer traffic.

For example, consider the estimates of the total cost of a 100MB/s workload on Managed Service for Apache Kafka in us-central1, assuming 40% average CPU utilization and 24 hour message storage. For comparison we also provide an estimate of what it would take to operate the same workload with an open source Apache Kafka cluster on Compute Engine using N2 CPUs and local replica followers for consumers.

Note that the relatively high change in the price with fanout is due to the PSC data transfer charges.