Create custom query quotas

This document describes how to set or modify custom query quotas to control costs. To learn how BigQuery analysts can estimate and control costs, see Estimate and control costs.

If you have multiple BigQuery projects and users, you can manage costs by requesting a custom quota that specifies a limit on the amount of data processed per day. Daily quotas are reset at midnight Pacific Time.

Custom quota is proactive, so you can't run an 11 TB query if you have a 10 TB quota. Creating a custom quota on processed data lets you control costs at the project level or at the user level.

  • QueryUsagePerDay: Project-level custom quotas limit the aggregate usage of all users in that project.

  • QueryUsagePerUserPerDay: User-level custom quota is separately applied to all users and service accounts within a project.

To set a custom quota, you need specific roles or permissions. For quotas to set, see Quotas and limits.

Required role

To get the permission that you need to change your quota, ask your administrator to grant you the Quota Administrator (role/servicemanagement.quotaAdmin) IAM role on your project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the serviceusage.quotas.update permission, which is required to change your quota.

You might also be able to get this permission with custom roles or other predefined roles.

Set or modify custom quotas

You can set a custom quota or modify an existing custom quota for any quota displayed on the Quotas page of Google Cloud console. For more information, see View and manage quotas. When you request a lower quota, the change takes effect within a few minutes. (If you request a higher quota, your request goes through an approval process, which can take more time. For more information, see Requesting a higher quota limit.)

You can view your custom quota levels on the Google Cloud console page. After you set a custom quota, BigQuery returns an error when you exceed it:

  • If you exceed a project-level custom quota, BigQuery returns the usageQuotaExceeded error:

    Custom quota exceeded: Your usage exceeded the custom quota for
QueryUsagePerDay, which is set by your administrator. For more information,
see https://cloud.google.com/bigquery/cost-controls

  • If the user exceeds a user-level custom quota, BigQuery returns a usageQuotaExceeded error with a different error message:

    Custom quota exceeded: Your usage exceeded the custom quota for
QueryUsagePerUserPerDay, which is set by your administrator. For more
information, see https://cloud.google.com/bigquery/cost-controls

You can run your query from another project that has access to your datasets and that doesn't have a custom quota or hasn't yet exceeded it.

Example

Suppose you set the following custom quotas for a project with 10 users, one of which is a service account:

  • Project level: 50 TB per day
  • User level: 10 TB per day

Project-level custom quotas limit the aggregate usage of all users in that project. User-level custom quotas are separately applied to each user or service account within a project.

The following table describes the remaining quota as the 10 users run queries throughout the day.

Usage Remaining quota
Each of the 10 users queries 4 TBs Project level: 10 TBs remain.
User level: 6 TBs per user remain, but only up to 10 TBs total.
The service account queries another 6 TBs Project level: 4 TBs remain.
User level: The service account can no longer use BigQuery. 6 TBs per user remain for the other users, but only up to 4 TBs total.
One user queries another 4 TBs Project level: 0 TBs remain.
User level: Various TBs remain, but no one can use BigQuery because the project-level quota has been exceeded.

With no remaining quota, BigQuery stops working for everyone in that project.

What's next