Stay organized with collections
Save and categorize content based on your preferences.
Introduction to search in BigQuery
BigQuery search indexes let you use GoogleSQL to
efficiently find
unique data elements that are buried in unstructured text and semi-structured
JSON data, without having to know the table schemas in advance.
With search indexes, BigQuery provides a powerful columnar store
and text search in one platform, enabling efficient row lookups when you need to
find individual rows of data. A common use case is log analytics. For example,
you might want to identify the rows of data associated with a user for General
Data Protection Regulation (GDPR) reporting, or to find specific error codes in
a text payload.
BigQuery stores and manages your indexes, so that when data becomes
available in BigQuery, you can immediately retrieve it with the
SEARCH function
or other operators and functions,
such as the equal (=), IN, or LIKE operators and certain string and JSON
functions. To optimize your searches, read about
best practices.
Use cases
BigQuery search indexes help you perform the following tasks:
Search system, network, or application logs stored in BigQuery
tables.
Identify data elements for deletion to comply with regulatory processes.
Support developer troubleshooting.
Perform security audits.
Create a dashboard that requires highly selective search filters.
There is no charge for the processing required to build and refresh your search
indexes when the total size of indexed tables in your organization is below
your region's
limit. To support indexing beyond this limit,
you need to
provide your own reservation
for handling the index-management jobs.
Search indexes incur storage costs when they are active.
You can find the index storage size in the
INFORMATION_SCHEMA.SEARCH_INDEXES view.
Roles and permissions
To create a search index, you need the
bigquery.tables.createIndex IAM permission
on the table where you're creating the index. To drop a search index, you need
the bigquery.tables.deleteIndex permission. Each of the following predefined
IAM roles includes the permissions that you need to work with
search indexes:
BigQuery Data Owner (roles/bigquery.dataOwner)
BigQuery Data Editor (roles/bigquery.dataEditor)
BigQuery Admin (roles/bigquery.admin)
Limitations
You can't create a search index directly on a view or materialized view, but
calling the
SEARCH function
on a view of an indexed table makes use of the underlying search index.
You can't create a search index on an external table.
If you rename a table after you create a search index on it, the index
becomes invalid.
The SEARCH function is designed for point lookups. Fuzzy searching,
typo correction, wildcards, and other types of document searches are not
available.
If the search index is not yet at 100% coverage, you are still charged for all
index storage that is reported in the
INFORMATION_SCHEMA.SEARCH_INDEXES view.
Queries that use the SEARCH function or are optimized by search indexes
are not accelerated by BigQuery BI Engine.
Search indexes are not used when the indexed table is modified by a DML
statement, but they can be used when the predicate that is optimizable by
search indexes is part of a subquery in a DML statement.
A search index is not used in the following query:
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-29 UTC."],[[["\u003cp\u003eBigQuery search indexes enable efficient searching of unstructured text and semi-structured JSON data using GoogleSQL, even without prior knowledge of table schemas.\u003c/p\u003e\n"],["\u003cp\u003eThese indexes facilitate row lookups, making them useful for tasks such as log analytics, GDPR compliance, and identifying specific error codes.\u003c/p\u003e\n"],["\u003cp\u003eBigQuery manages the indexes, allowing for immediate data retrieval through the \u003ccode\u003eSEARCH\u003c/code\u003e function and other operators like \u003ccode\u003e=\u003c/code\u003e, \u003ccode\u003eIN\u003c/code\u003e, or \u003ccode\u003eLIKE\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eUsing search indexes for indexed tables below a certain size limit in your region incurs no processing costs for building or refreshing the index, however storage costs for active indexes do apply.\u003c/p\u003e\n"],["\u003cp\u003eSearch indexes have limitations, they cannot be directly applied to views or materialized views, they do not allow fuzzy searching or typo correction, and aren't utilized during DML statements on the indexed table itself, among other limitations.\u003c/p\u003e\n"]]],[],null,["# Introduction to search in BigQuery\n==================================\n\n| **Note:** This feature may not be available when using reservations that are created with certain BigQuery editions. For more information about which features are enabled in each edition, see [Introduction to\n| BigQuery editions](/bigquery/docs/editions-intro).\n\nBigQuery search indexes let you use GoogleSQL to\nefficiently find\nunique data elements that are buried in unstructured text and semi-structured\nJSON data, without having to know the table schemas in advance.\n\nWith search indexes, BigQuery provides a powerful columnar store\nand text search in one platform, enabling efficient row lookups when you need to\nfind individual rows of data. A common use case is log analytics. For example,\nyou might want to identify the rows of data associated with a user for General\nData Protection Regulation (GDPR) reporting, or to find specific error codes in\na text payload.\n\nBigQuery stores and manages your indexes, so that when data becomes\navailable in BigQuery, you can immediately retrieve it with the\n[`SEARCH` function](/bigquery/docs/reference/standard-sql/search_functions#search)\nor [other operators and functions](/bigquery/docs/search#operator_and_function_optimization),\nsuch as the equal (`=`), `IN`, or `LIKE` operators and certain string and JSON\nfunctions. To optimize your searches, read about\n[best practices](/bigquery/docs/search#best_practices).\n| **Important:** Join the [Search discussion group](https://groups.google.com/g/bq-search) to post questions and comments, and to follow the latest updates.\n\nUse cases\n---------\n\nBigQuery search indexes help you perform the following tasks:\n\n- Search system, network, or application logs stored in BigQuery tables.\n- Identify data elements for deletion to comply with regulatory processes.\n- Support developer troubleshooting.\n- Perform security audits.\n- Create a dashboard that requires highly selective search filters.\n- Search pre-processed data for exact matches.\n\nFor more information, see\n[Create a search index](/bigquery/docs/search-index) and\n[Search with an index](/bigquery/docs/search).\n\nPricing\n-------\n\nThere is no charge for the processing required to build and refresh your search\nindexes when the total size of indexed tables in your organization is below\nyour region's\n[limit](/bigquery/quotas#index_limits). To support indexing beyond this limit,\nyou need to\n[provide your own reservation](/bigquery/docs/search-index#use_your_own_reservation)\nfor handling the index-management jobs.\nSearch indexes incur storage costs when they are active.\nYou can find the index storage size in the\n[`INFORMATION_SCHEMA.SEARCH_INDEXES` view](/bigquery/docs/information-schema-indexes).\n\nRoles and permissions\n---------------------\n\nTo create a search index, you need the\n[`bigquery.tables.createIndex` IAM permission](/bigquery/docs/access-control#bq-permissions)\non the table where you're creating the index. To drop a search index, you need\nthe `bigquery.tables.deleteIndex` permission. Each of the following predefined\nIAM roles includes the permissions that you need to work with\nsearch indexes:\n\n- BigQuery Data Owner (`roles/bigquery.dataOwner`)\n- BigQuery Data Editor (`roles/bigquery.dataEditor`)\n- BigQuery Admin (`roles/bigquery.admin`)\n\nLimitations\n-----------\n\n- You can't create a search index directly on a view or materialized view, but calling the [`SEARCH` function](/bigquery/docs/reference/standard-sql/search_functions#search) on a view of an indexed table makes use of the underlying search index.\n- You can't create a search index on an external table.\n- If you rename a table after you create a search index on it, the index becomes invalid.\n- The `SEARCH` function is designed for point lookups. Fuzzy searching, typo correction, wildcards, and other types of document searches are not available.\n- If the search index is not yet at 100% coverage, you are still charged for all index storage that is reported in the [`INFORMATION_SCHEMA.SEARCH_INDEXES` view](/bigquery/docs/information-schema-indexes).\n- Queries that use the `SEARCH` function or are optimized by search indexes are not accelerated by [BigQuery BI Engine](/bigquery/docs/bi-engine-intro).\n- Search indexes are not used when the indexed table is modified by a DML\n statement, but they can be used when the predicate that is optimizable by\n search indexes is part of a subquery in a DML statement.\n\n - A search index is not used in the following query:\n\n ```googlesql\n DELETE FROM my_dataset.indexed_table\n WHERE SEARCH(user_id, '123');\n ```\n - A search index can be used in the following query:\n\n ```googlesql\n DELETE FROM my_dataset.other_table\n WHERE\n user_id IN (\n SELECT user_id\n FROM my_dataset.indexed_table\n WHERE SEARCH(user_id, '123')\n );\n ```\n- Search indexes are not used when the query references [Materialized Views](/bigquery/docs/materialized-views-intro).\n\n- Search indexes are not used in a [multi-statement transaction query](/bigquery/docs/transactions).\n\n- Search indexes are not used in a [time-travel query](/bigquery/docs/time-travel).\n\nWhat's next\n-----------\n\n- Learn more about [creating a search index](/bigquery/docs/search-index).\n- Learn more about [searching in a table with a search index](/bigquery/docs/search)."]]