Create custom query quotas
This document describes how to set or modify custom query quotas to control costs. To learn how BigQuery analysts can estimate and control costs, see Estimate and control costs.
If you have multiple BigQuery projects and users, you can manage costs by requesting a custom quota that specifies a limit on the amount of data processed per day. Daily quotas are reset at midnight Pacific Time.
Custom quota is proactive, so you can't run an 11 TB query if you have a 10 TB quota. Creating a custom quota on processed data lets you control costs at the project level or at the user level.
To set custom cost controls, you can update one or both of the following query quotas:
QueryUsagePerDay
: Project-level custom quotas limit the aggregate usage of all users in that project.QueryUsagePerUserPerDay
: User-level custom quota is separately applied to all users and service accounts within a project.
For more information about BigQuery quotas that you can set, see Quotas and limits.
Required role
To get the permission that
you need to change your quota,
ask your administrator to grant you the
Quota Administrator (role/servicemanagement.quotaAdmin
)
IAM role on your project.
For more information about granting roles, see Manage access to projects, folders, and organizations.
This predefined role contains the
serviceusage.quotas.update
permission,
which is required to
change your quota.
You might also be able to get this permission with custom roles or other predefined roles.
Set or modify custom quotas
You can set a custom quota or modify an existing custom quota for any quota displayed on the Quotas & System Limits page of the Google Cloud console. When you request a lower quota, the change takes effect within a few minutes. If you request a higher quota, your request goes through an approval process, which can take more time. For more information, see Request a quota adjustment.
To set or update a custom cost control, such as limiting the amount of BigQuery data that can be processed each day, do the following:
In the Google Cloud console, open the IAM & Admin > Quotas & System Limits page:
Use the Service filter in the Filter search box to filter for the BigQuery API.
Select the quotas that you want to adjust. For example, to limit the amount of data that be queried per day at both the project level and user level, select Query usage per day and Query usage per day per user. You might need to page through the list to find them. After you select a quota, a toolbar appears.
In the toolbar, click
Edit. The Quota changes dialog opens.If Unlimited is selected, deselect it.
Enter the quota value in TiB that you want in the New value field.
Click Done.
Click Submit request.
For more information about viewing and managing quotas, see View and manage quotas.
Returned error messages
After you set a custom quota, BigQuery returns an error when you exceed it:
If you exceed a project-level custom quota, BigQuery returns the
usageQuotaExceeded
error:Custom quota exceeded: Your usage exceeded the custom quota for QueryUsagePerDay, which is set by your administrator. For more information, see https://cloud.google.com/bigquery/cost-controls
If the user exceeds a user-level custom quota, BigQuery returns a
usageQuotaExceeded
error with a different error message:Custom quota exceeded: Your usage exceeded the custom quota for QueryUsagePerUserPerDay, which is set by your administrator. For more information, see https://cloud.google.com/bigquery/cost-controls
You can run your query from another project that has access to your datasets and that doesn't have a custom quota or hasn't yet exceeded it.
Example
Suppose you set the following custom quotas for a project with 10 users, one of which is a service account:
- Project level: 50 TB per day
- User level: 10 TB per day
Project-level custom quotas limit the aggregate usage of all users in that project. User-level custom quotas are separately applied to each user or service account within a project.
The following table describes the remaining quota as the 10 users run queries throughout the day.
Usage | Remaining quota |
---|---|
Project level: 10 TBs remain. User level: 6 TBs per user remain, but only up to 10 TBs total. |
|
The service account queries another 6 TBs | Project level: 4 TBs remain. User level: The service account can no longer use BigQuery. 6 TBs per user remain for the other users, but only up to 4 TBs total. |
One user queries another 4 TBs | Project level: 0 TBs remain. User level: Various TBs remain, but no one can use BigQuery because the project-level quota has been exceeded. |
With no remaining quota, BigQuery stops working for everyone in that project.
What's next
- Learn about BigQuery pricing.
- Learn how to estimate and control costs.
- Learn how to analyze BigQuery audit logs to monitor query costs and BigQuery usage.
To learn about billing, alerts, and visualizing data, see the following topics: