O Google SecOps oferece um data lake gerenciado de telemetria normalizada e enriquecida com informações sobre ameaças exportando dados para o BigQuery. Isso permite que você faça o
seguinte:
Executar consultas ad hoc diretamente no BigQuery.
Use suas próprias ferramentas de Business Intelligence, como o Looker ou o Microsoft
Power BI, para criar painéis, relatórios e análises.
Combine dados do Google SecOps com conjuntos de dados de terceiros.
Execute análises usando ferramentas de ciência de dados ou machine learning.
Gerar relatórios usando painéis padrão predefinidos e personalizados.
O Google SecOps exporta as seguintes categorias de dados para o BigQuery:
Registros de eventos da UDM:registros da UDM criados com base em dados de registro ingeridos pelos clientes.
Esses registros são enriquecidos com informações de alias.
Correspondências de regras (detecções): instâncias em que uma regra corresponde a um ou mais eventos.
Correspondências de IoC: artefatos (por exemplo, domínios, endereços IP) de eventos que
correspondem a feeds de indicadores de comprometimento (IoC). Isso inclui correspondências de feeds globais e específicos do cliente.
Métricas de ingestão:incluem estatísticas, como número de linhas de registro ingeridas, número de eventos produzidos com base em registros, número de erros de registro indicando que os registros não puderam ser analisados e o estado dos encaminhadores do Google SecOps.
Para mais informações, consulte Esquema do BigQuery para métricas de ingestão.
Gráfico de entidades e relações entre entidades: armazena a descrição de entidades e as relações delas com outras entidades.
Visão geral das tabelas
O Google SecOps cria o conjunto de dados datalake no BigQuery e as seguintes tabelas:
entity_enum_value_to_name_mapping: para tipos enumerados na tabela entity_graph, mapeia os valores numéricos para os valores de string.
entity_graph: armazena dados sobre entidades do UDM.
ingestion_metrics:
armazena estatísticas relacionadas à ingestão e normalização de dados de fontes de ingestão específicas, como encaminhadores, feeds e API Ingestion do Google SecOps.
ioc_matches: armazena correspondências de IOC encontradas em eventos da UDM.
job_metadata: uma tabela interna usada para rastrear a exportação de dados para o BigQuery.
rule_detections: armazena detecções retornadas por regras executadas no Google SecOps.
rulesets: armazena informações sobre detecções selecionadas do Google SecOps,
incluindo a categoria a que cada conjunto de regras pertence, se está ativado e
o status atual de alerta.
udm_enum_value_to_name_mapping: para tipos enumerados na tabela de eventos, mapeia os valores numéricos para os valores de string.
udm_events_aggregates: armazena dados agregados resumidos por hora de eventos normalizados.
Acessar dados no BigQuery
Você pode executar consultas diretamente no BigQuery ou conectar sua própria ferramenta de inteligência de negócios, como o Looker ou o Microsoft Power BI, ao BigQuery.
Para ativar o acesso à instância do BigQuery, use a CLI do Google SecOps ou a API de acesso ao BigQuery do Google SecOps.
Você pode fornecer um endereço de e-mail para um usuário ou um grupo de sua propriedade. Se você
configurar o acesso a um grupo, use-o para gerenciar quais membros da equipe podem
acessar a instância do BigQuery.
Para conectar o Looker ou outra ferramenta de business intelligence ao BigQuery, entre em contato com seu representante do Google SecOps para receber as credenciais da conta de serviço que permitem conectar um aplicativo ao conjunto de dados do BigQuery do Google SecOps. A conta de serviço terá o papel de leitor de dados do BigQuery do IAM (roles/bigquery.dataViewer) e de leitor de jobs do BigQuery (roles/bigquery.jobUser).
[[["Fácil de entender","easyToUnderstand","thumb-up"],["Meu problema foi resolvido","solvedMyProblem","thumb-up"],["Outro","otherUp","thumb-up"]],[["Difícil de entender","hardToUnderstand","thumb-down"],["Informações incorretas ou exemplo de código","incorrectInformationOrSampleCode","thumb-down"],["Não contém as informações/amostras de que eu preciso","missingTheInformationSamplesINeed","thumb-down"],["Problema na tradução","translationIssue","thumb-down"],["Outro","otherDown","thumb-down"]],["Última atualização 2025-08-21 UTC."],[[["\u003cp\u003eGoogle Security Operations exports normalized and enriched telemetry data to BigQuery, enabling users to perform ad-hoc queries, use business intelligence tools, join with third-party datasets, and run advanced analytics.\u003c/p\u003e\n"],["\u003cp\u003eFrom December 31, 2024, only Enterprise Plus Tier customers will have access to the managed BigQuery data lake, with the managed resources and API keys fully deprecated by March 31, 2025, for others.\u003c/p\u003e\n"],["\u003cp\u003eThe exported data includes UDM event records, rule matches, IoC matches, ingestion metrics, and entity graph data, all stored in a customer-specific Google Cloud project managed by Google.\u003c/p\u003e\n"],["\u003cp\u003eData is exported on a fill-forward basis with a 365-day retention period, and raw logs are not exported to the Google Security Operations data lake in BigQuery.\u003c/p\u003e\n"],["\u003cp\u003eAccess to the BigQuery instance can be managed through the Google Security Operations CLI or API, and service account credentials for Looker and other BI tools can be obtained from a Google Security Operations representative.\u003c/p\u003e\n"]]],[],null,["# Google SecOps data in BigQuery\n==============================\n\nSupported in: \nGoogle secops [SIEM](/chronicle/docs/secops/google-secops-siem-toc)\n| **Note:** This option is available for Google SecOps Enterprise Plus customers only. For all other customers, see [Configure data export to BigQuery in a self-managed Google Cloud project](/chronicle/docs/preview/cloud-integration/export-to-customer-managed-project).\n\nGoogle SecOps provides a managed data lake of normalized and threat intelligence enriched\ntelemetry by exporting data to BigQuery. This lets you do the\nfollowing:\n\n- Run ad-hoc queries directly in BigQuery.\n- Use your own business intelligence tools, such as Looker or Microsoft Power BI, to create dashboards, reports, and analytics.\n- Join Google SecOps data with third-party datasets.\n- Run analytics using data science or machine learning tools.\n- Run reports using predefined default dashboards and custom dashboards.\n\nGoogle SecOps exports the following categories of data to BigQuery:\n\n- **UDM event records:** UDM records created from log data ingested by customers. These records are enriched with aliasing information.\n- **Rules matches (detections)**: instances where a rule matches one or more events.\n- **IoC matches**: artifacts (for example domains, IP addresses) from events that match Indicator of Compromise (IoC) feeds. This includes matches to from global feeds and customer-specific feeds.\n- **Ingestion metrics:** include statistics, such as number of log lines ingested, number of events produced from logs, number of log errors indicating that logs couldn't be parsed, and the state of Google SecOps forwarders. For more information, see [Ingestion metrics BigQuery schema](/chronicle/docs/reference/ingestion-metrics-schema).\n- **Entity graph and entity relationships**: stores the description of entities and their relationships with other entities.\n\nOverview of the tables\n----------------------\n\nGoogle SecOps creates the `datalake` dataset in BigQuery and the following tables:\n\n- `entity_enum_value_to_name_mapping`: for enumerated types in the `entity_graph` table, maps the numerical values to the string values.\n- `entity_graph`: stores data about UDM entities.\n- [`events`](/chronicle/docs/reports/events-schema-overview): stores data about UDM events.\n- [`ingestion_metrics`](/chronicle/docs/reference/ingestion-metrics-schema): stores statistics related to ingestion and normalization of data from specific ingestion sources, such as Google SecOps forwarders, feeds, and Ingestion API.\n- `ioc_matches`: stores IOC matches found against UDM events.\n- `job_metadata`: an internal table used to track the export of data to BigQuery.\n- `rule_detections`: stores detections returned by rules run in Google SecOps.\n- `rulesets`: stores information about Google SecOps curated detections, including the category each rule set belongs to, whether it is enabled, and the current alerting status.\n- `udm_enum_value_to_name_mapping`: For enumerated types in the events table, maps the numerical values to the string values.\n- `udm_events_aggregates`: stores aggregated data summarized by hour of normalized events.\n\nAccess data in BigQuery\n-----------------------\n\nYou can run queries directly in BigQuery or connect your own business\nintelligence tool, such as Looker or Microsoft Power BI, to BigQuery.\n\nTo enable access to the BigQuery instance, use the\n[Google SecOps BigQuery Access API](/chronicle/docs/reference/bigquery-access-api#access_api_reference).\nYou can provide an email address for either a user or a group that you own. If you\nconfigure access to a group, use the group to manage which team members can\naccess the BigQuery instance.\n\nTo connect Looker or another business intelligence tool to BigQuery, contact\nyour Google SecOps representative for service account credentials that enable you to\nconnect an application to the Google SecOps BigQuery dataset. The service\naccount will have IAM BigQuery Data Viewer role (`roles/bigquery.dataViewer`) and BigQuery Job Viewer role (`roles/bigquery.jobUser`).\n\nWhat's next\n-----------\n\n- Learn more about the following schemas:\n - [`events`](/chronicle/docs/reports/events-schema-overview)\n - [`ingestion_metrics`](/chronicle/docs/reference/ingestion-metrics-schema)\n- For information about accessing and running queries in BigQuery, see [Run interactive and batch query jobs](/bigquery/docs/running-queries).\n- For information about how to query partitioned tables, see [Query partitioned tables](/bigquery/docs/querying-partitioned-tables).\n- For information about how to connect Looker to BigQuery, see Looker documentation about [connecting to BigQuery](/looker/docs/db-config-google-bigquery).\n\n**Need more help?** [Get answers from Community members and Google SecOps professionals.](https://security.googlecloudcommunity.com/google-security-operations-2)"]]
