Organization Policy provides predefined constraints for various Google Cloud services. However, if you want more granular, customizable control over the specific fields that are restricted in your organization policies, you can also create custom organization policies.
Benefits
You can use a custom organization policy to allow or deny the creation of data lineage processes with conditions based on supported resource attributes, such as process name, source type, and origin.
Policy inheritance
By default, organization policies are inherited by the descendants of the resources on which you enforce the policy. For example, if you enforce a policy on a folder, Google Cloud enforces the policy on all projects in the folder. To learn more about this behavior and how to change it, refer to Hierarchy evaluation rules.
Pricing
The Organization Policy Service, including predefined and custom organization policies, is offered at no charge.
Limitations
Custom constraints can be enforced only for data lineage
Process
resources. Other resources, such asRuns
andEvents
, aren't supported.Newly enforced custom constraints don't apply to existing resources.
Before you begin
- Ensure that you know your organization ID.
-
If you want to test custom organization policies that reference data lineage resources, create a new project. Testing these organization policies in an existing project could disrupt security workflows.
-
In the Google Cloud console, go to the project selector page.
-
Click Create project.
-
Name your project. Make a note of your generated project ID.
-
Edit the other fields as needed.
-
Click Create.
-
Required roles
To get the permissions that
you need to manage organization policies,
ask your administrator to grant you the
Organization Policy Administrator (roles/orgpolicy.policyAdmin
) IAM role on the organization.
For more information about granting roles, see Manage access to projects, folders, and organizations.
This predefined role contains the permissions required to manage organization policies. To see the exact permissions that are required, expand the Required permissions section:
Required permissions
The following permissions are required to manage organization policies:
-
orgpolicy.constraints.list
-
orgpolicy.policies.create
-
orgpolicy.policies.delete
-
orgpolicy.policies.list
-
orgpolicy.policies.update
-
orgpolicy.policy.get
-
orgpolicy.policy.set
You might also be able to get these permissions with custom roles or other predefined roles.
Create a custom constraint
A custom constraint is defined in a YAML file by the resources, methods, conditions, and actions that are supported by the service on which you are enforcing the organization policy. Conditions for your custom constraints are defined using Common Expression Language (CEL). For more information about how to build conditions in custom constraints using CEL, see the CEL section of Creating and managing custom constraints.
To create a YAML file for a custom constraint:
name: organizations/ORGANIZATION_ID/customConstraints/CONSTRAINT_NAME
resourceTypes:
- datalineage.googleapis.com/RESOURCE_TYPE
methodTypes:
- CREATE
- UPDATE
condition: "CONDITION"
actionType: ACTION
displayName: DISPLAY_NAME
description: DESCRIPTION
Replace the following:
ORGANIZATION_ID
: your organization ID, such as123456789
.CONSTRAINT_NAME
: the name you want for your new custom constraint. A custom constraint must start withcustom.
, and can only include uppercase letters, lowercase letters, or numbers—for example,custom.denyLineageProcess
. The maximum length of this field is 70 characters, not counting the prefix—for example,organizations/123456789/customConstraints/custom
.RESOURCE_TYPE
: the name (not the URI) of the Data Lineage API REST resource containing the object and field you want to restrict. For data lineage onlyProcess
is available.CONDITION
: a CEL condition that is written against a representation of a supported service resource. This field has a maximum length of 1000 characters. See Supported resources for more information about the resources available to write conditions against. For example,"resource.name.contains('invalid_name')"
.ACTION
: the action to take if thecondition
is met. This can be eitherALLOW
orDENY
.DISPLAY_NAME
: a human-friendly name for the constraint. This field has a maximum length of 200 characters.DESCRIPTION
: a human-friendly description of the constraint to display as an error message when the policy is violated. This field has a maximum length of 2000 characters.
For more information about how to create a custom constraint, see Defining custom constraints.
Set up a custom constraint
After you have created the YAML file for a new custom constraint, you must set it up to make it available for organization policies in your organization. To set up a custom constraint, use thegcloud org-policies set-custom-constraint
command:
gcloud org-policies set-custom-constraint CONSTRAINT_PATH
CONSTRAINT_PATH
with the full path to your
custom constraint file. For example, /home/user/customconstraint.yaml
.
Once completed, your custom constraints are available as organization policies
in your list of Google Cloud organization policies.
To verify that the custom constraint exists, use the
gcloud org-policies list-custom-constraints
command:
gcloud org-policies list-custom-constraints --organization=ORGANIZATION_ID
ORGANIZATION_ID
with the ID of your organization resource.
For more information, see
Viewing organization policies.
Enforce a custom organization policy
You can enforce a boolean constraint by creating an organization policy that references it, and then applying that organization policy to a Google Cloud resource.Console
- In the Google Cloud console, go to the Organization policies page.
- From the project picker, select the project for which you want to set the organization policy.
- From the list on the Organization policies page, select your constraint to view the Policy details page for that constraint.
- To configure the organization policy for this resource, click Manage policy.
- On the Edit policy page, select Override parent's policy.
- Click Add a rule.
- In the Enforcement section, select whether enforcement of this organization policy is on or off.
- Optional: To make the organization policy conditional on a tag, click Add condition. Note that if you add a conditional rule to an organization policy, you must add at least one unconditional rule or the policy cannot be saved. For more information, see Setting an organization policy with tags.
- If this is a custom constraint, you can click Test changes to simulate the effect of this organization policy. For more information, see Test organization policy changes with Policy Simulator.
- To finish and apply the organization policy, click Set policy. The policy requires up to 15 minutes to take effect.
gcloud
To create an organization policy that enforces a boolean constraint, create a policy YAML file that references the constraint:
name: projects/PROJECT_ID/policies/CONSTRAINT_NAME spec: rules: - enforce: true
Replace the following:
-
PROJECT_ID
: the project on which you want to enforce your constraint. -
CONSTRAINT_NAME
: the name you defined for your custom constraint. For example,custom.denyLineageProcess
.
To enforce the organization policy containing the constraint, run the following command:
gcloud org-policies set-policy POLICY_PATH
Replace POLICY_PATH
with the full path to your organization policy
YAML file. The policy requires up to 15 minutes to take effect.
Test the custom organization policy
This section describes how to test an organization policy constraint that restricts users from creating a data lineage process with a name that contains the text invalid_name
.
If you want to test this custom constraint, do the following:
Create a new project, as described in the Before you begin section.
Copy the following constraint into a YAML file:
name: organizations/ORGANIZATION_ID/customConstraints/custom.denyLineageProcess resourceTypes: datalineage.googleapis.com/Process methodTypes: - CREATE - UPDATE condition: "resource.name.contains('invalid_name')" actionType: DENY displayName: Do not allow data lineage process with 'invalid_name' to be created.
Replace
ORGANIZATION_ID
with the numeric ID of your Google Cloud organization.Set up the custom constraint and enforce it for the project that you created to test the custom organization policy constraint.
Create a process with the name that you included in the custom constraint.
curl -s -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application.json" \ "https://datalineage.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processes" \ -d '{"name":"projects/PROJECT_ID/locations/LOCATION/processes/invalid_name"}'
Replace the following:
PROJECT_ID
: ID of the project where the process is created. It must be in the same organization that the constraint is defined in.LOCATION
: Google Cloud region where the process is created.
The output is the following:
Operation denied by org policy on resource 'projects/PROJECT_ID/locations/LOCATION': ["customConstraints/custom.denyLineageProcess"]
Data lineage supported resources and operations
The following custom constraint fields are available to use when you create or update a data lineage process:
resource.name
resource.displayName
resource.origin.name
resource.origin.sourceType
Custom constraints are checked for the following methods:
Example custom organization policies for common use cases
The following table provides the syntax of some custom constraints for common use cases:
For more information about CEL macros available for use in custom constraint conditions, see Common Expression Language.
Use case | Constraint syntax |
---|---|
Disable the creation of data lineage processes |
name: organizations/ORGANIZATION_ID/customConstraints/custom.denyLineageProcessesCreation resourceTypes: - datalineage.googleapis.com/Process methodTypes: - CREATE condition: "True" actionType: DENY displayName: Deny creation of all data lineage processes. |
Disable the creation of data lineage processes by BigQuery or Dataproc |
name: organizations/ORGANIZATION_ID/customConstraints/custom.denyLineageProcessesCreation resourceTypes: - datalineage.googleapis.com/Process methodTypes: - CREATE condition: "resource.origin.sourceType == 'BIGQUERY' || resource.origin.sourceType == 'DATAPROC'" actionType: DENY displayName: Deny data lineage processes created by BigQuery or Dataproc. |
Disable data lineage processes with the specified name |
name: organizations/ORGANIZATION_ID/customConstraints/custom.denyLineageProcessesCreation resourceTypes: - datalineage.googleapis.com/Process methodTypes: - CREATE - UPDATE condition: "resource.name.contains('RESTRICTED_NAME')" actionType: DENY displayName: Deny data lineage processes whose name contains RESTRICTED_NAME. |
What's next
- See Introduction to the Organization Policy Service to learn more about organization policies.
- Learn more about how to create and manage organization policies.
- See the full list of predefined organization policy constraints.