Create a Managed Kafka ACL

Stay organized with collections Save and categorize content based on your preferences.

Managed Kafka access control lists (ACLs) provide fine-grained authorization for operations within your Managed Service for Apache Kafka cluster. Creating a Managed Kafka ACL resource defines a set of permission rules for a specific Kafka resource pattern. For foundational concepts, see the Access control with IAM and Kafka ACLs.

Before you begin

Ensure you have an existing Managed Service for Apache Kafka cluster.

Required roles and permissions

To get the permissions that you need to create a Managed Kafka ACL, ask your administrator to grant you the Managed Kafka ACL Editor (roles/managedkafka.aclEditor) IAM role on your project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the permissions required to create a Managed Kafka ACL. To see the exact permissions that are required, expand the Required permissions section:

You might also be able to get these permissions with custom roles or other predefined roles.

The Managed Kafka ACL Editor (roles/managedkafka.aclEditor) role grants permissions to create, view, update, and delete ACL resources. For more information about the role, see Managed Service for Apache Kafka predefined roles.

Properties of a Managed Kafka ACL

When you create a Managed Kafka ACL resource, you must define the following properties:

ACL ID

This is the unique identifier for the Managed Kafka ACL resource within the cluster and location. The structure of ACL_ID also defines the resource pattern (resource type, resource name, pattern type) to which the ACL rules apply.

Choose an ACL_ID based on the resource pattern you want to target for the ACL binding:

• Cluster: cluster (Applies to the cluster resource itself)

• Specific topic: topic/{topic_name}

• Specific group: consumerGroup/{group_name}

• Specific transactional ID: transactionalId/{transactional_id}

• All topics with a prefix: topicPrefixed/{prefix}

• All consumer groups with a prefix: consumerGroupPrefixed/{prefix}

• All transactional IDs with a prefix: transactionalIdPrefixed/{prefix}

• All topics: allTopics (Represents topic/*)

• All groups: allConsumerGroups (Represents consumerGroup/*)

• All transactional IDs: allTransactionalIds (Represents transactionalId/*)

For more information, see the Guidelines to name a Managed Service for Apache Kafka resource.

You cannot change the ACL ID after creation. It is immutable.

ACL entries

This is the list of specific permission rules that apply to the resource pattern defined by the ACL ID.

• You must provide at least one ACL entry when creating the resource; an empty list is not allowed.

• The maximum number of allowed entries per Managed Kafka ACL resource is 100. If you need more rules for a single resource pattern, you need to manage ACLs directly using standard Apache Kafka tools.

• Each ACL entry contains the following fields:

  • Principal: the user or service account to which the rule applies. Specify using the format User:{google_service_account_email} or use the wildcard User:*.

  • Operation: the Kafka operation type, such as ALL, READ, WRITE, CREATE, DELETE, ALTER, DESCRIBE, CLUSTER_ACTION, DESCRIBE_CONFIGS, ALTER_CONFIGS, or IDEMPOTENT_WRITE (case-insensitive). For valid operation and resource type combinations, see the Apache Kafka documentation.

  • Permission-type: one of ALLOW or DENY (case-insensitive).

  • Host: must be set to the wildcard * for Managed Service for Apache Kafka.

You can provide these entries directly on the command line using the --acl-entry flag or by supplying a JSON or YAML file using the --acl-entries-from-file flag.

Output-only resource pattern properties

When you create an ACL, Managed Service for Apache Kafka populates the following resource pattern fields based on the provided ACL ID:

• Resource type: the ACL resource type derived from the name. One of: CLUSTER, TOPIC, GROUP, or TRANSACTIONAL_ID.

• Resource name: the ACL resource name (or matching prefix, for PREFIXED pattern type) derived from the name. For CLUSTER resource type, this is always "kafka-cluster". Can be the wildcard literal "*".

• ACL pattern type: the ACL pattern type derived from the name. One of: LITERAL or PREFIXED.

These fields are output-only and provided for convenience in mapping Managed Kafka ACL resources to Kafka ACL bindings.

Etag

The Etag is used for concurrency control. An Etag is returned in the response to GetAcl and CreateAcl operations. Callers are required to put that Etag in the request to UpdateAcl to ensure that their changes are applied to the same version of the ACL entries that exists in the Kafka Cluster.

A terminal 'T' character in the Etag indicates that the ACL entries are truncated. More entries for the ACL exist on the Kafka Cluster, but can't be returned in the ACL due to repeated field limits.

Create an ACL

1. Install the Google Cloud CLI.

2. If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

3. To initialize the gcloud CLI, run the following command:

   gcloud init

4. Run the gcloud managed-kafka acls create command:

   gcloud managed-kafka acls create ACL_ID \
     --cluster=CLUSTER_ID \
     --location=LOCATION \
     --acl-entries-from-file=PATH_TO_FILE | \
     --acl-entry=principal=PRINCIPAL,operation=OPERATION, \
     permission-type=PERMISSION-TYPE,host=HOST \

   Replace the following:

   • ACL_ID (required): the unique ID for the Managed Kafka ACL resource. This defines the resource pattern. For more information about the ACL ID, see ACL ID.
   • CLUSTER_ID (required): the ID of the cluster where the ACL resource is created.
   • LOCATION (required): the region where the cluster is located, such as us-central1. See Supported locations.
   • You must specify one of the following flags to provide the ACL entries:
     • PATH_TO_FILE (required): path to a local JSON or YAML file containing the list of ACL entries in the required format.
     • --acl-entry=principal=PRINCIPAL,operation=OPERATION, permission-type=PERMISSION-TYPE,host=HOST (required): specifies a single ACL entry directly. Repeat this flag for each entry you want to add.

Use --acl-entry

Run the gcloud managed-kafka acls create command to provide ACL entries directly. You can run the command multiple times to specify multiple ACL entries for an ACL.

gcloud managed-kafka acls create ACL_ID \
  --cluster=CLUSTER_ID \
  --location=LOCATION \
  --acl-entry=principal=PRINCIPAL,operation=OPERATION, \
  permission-type=PERMISSION-TYPE,host=HOST\

Replace the following:

• ACL_ID (required): the ID for the new ACL resource such as topic/my-topic or cluster.

• CLUSTER_ID (required): the ID of your cluster.

• LOCATION (required): the region of your cluster such as us-central1.

• --acl-entry (required): specifies one ACL entry. Repeat the flag for multiple entries.

  • PRINCIPAL_1, PRINCIPAL_2: the principal for the ACL entry such as 'User:admin@example.iam.gserviceaccount.comor 'User:*'.

  • OPERATION_1, OPERATION_2: the operation type such as ALLor READ.

  • PERMISSION_1, PERMISSION_2: the permission type such as ALLOW or DENY.

  • HOST_1, HOST_2: the host for the ACL entry. For Google Cloud Managed Service for Apache Kafka, this must be set to the wildcard '*'.

Use --acl-entries-from-file

1. Create a local file such as acl_entries.yaml with the required ACL entries in YAML format:

   ---
   aclEntries:
   - principal: User:admin@project.iam.gserviceaccount.com
     operation: READ
     permissionType: ALLOW
     host: "*"
   - principal: User:admin@project.iam.gserviceaccount.com
     operation: WRITE
     permissionType: ALLOW
     host: "*"
   

2. Run the gcloud managed-kafka acls create command, referencing the file:

   gcloud managed-kafka acls create ACL_ID \
     --cluster=CLUSTER_ID \
     --location=LOCATION \
     --acl-entries-from-file=acl_entries.yaml
   

   Replace the following:

   • ACL_ID (required): the ID for the new ACL resource such as topic/my-topic or cluster.

   • CLUSTER_ID (required): the ID of your cluster.

   • LOCATION (required): the region of your cluster such as us-central1.

   • acl_entries.yaml (required): the path to your local file containing the ACL entries list.

What's next?

• View an ACL

• Update an ACL

• Delete an ACL