Google SecOps menyediakan data lake terkelola yang berisi telemetri yang dinormalisasi dan diperkaya dengan informasi terkait ancaman keamanan dengan mengekspor data ke BigQuery. Hal ini memungkinkan Anda melakukan
hal berikut:
Jalankan kueri ad-hoc langsung di BigQuery.
Gunakan alat business intelligence Anda sendiri, seperti Looker atau Microsoft
Power BI, untuk membuat dasbor, laporan, dan analisis.
Gabungkan data Google SecOps dengan set data pihak ketiga.
Jalankan analisis menggunakan alat ilmu data atau machine learning.
Jalankan laporan menggunakan dasbor default standar dan dasbor kustom yang telah ditentukan sebelumnya.
Google SecOps mengekspor kategori data berikut ke BigQuery:
Catatan peristiwa UDM: Catatan UDM yang dibuat dari data log yang di-ingest oleh pelanggan.
Data ini dilengkapi dengan informasi alias.
Kecocokan aturan (deteksi): instance saat aturan cocok dengan satu atau beberapa peristiwa.
Kecocokan IoC: artefak (misalnya, domain, alamat IP) dari peristiwa yang cocok dengan feed Indikator Gangguan (IoC). Hal ini mencakup kecocokan dari feed global dan feed khusus pelanggan.
Metrik penyerapan: mencakup statistik, seperti jumlah baris log yang diserap, jumlah peristiwa yang dihasilkan dari log, jumlah error log yang menunjukkan bahwa log tidak dapat diuraikan, dan status penerusan Google SecOps.
Untuk mengetahui informasi selengkapnya, lihat Skema BigQuery metrik penyerapan.
Grafik entitas dan hubungan entitas: menyimpan deskripsi entitas dan hubungannya dengan entitas lain.
Ringkasan tabel
Google SecOps membuat set data datalake di BigQuery dan tabel berikut:
entity_enum_value_to_name_mapping: untuk jenis yang di-enum di tabel
entity_graph, memetakan nilai numerik ke nilai string.
ingestion_metrics:
menyimpan statistik terkait penyerapan dan normalisasi data dari sumber penyerapan tertentu, seperti penerusan Google SecOps, feed, dan Ingestion API.
ioc_matches: menyimpan kecocokan IOC yang ditemukan terhadap peristiwa UDM.
job_metadata: tabel internal yang digunakan untuk melacak ekspor data ke
BigQuery.
rule_detections: menyimpan deteksi yang ditampilkan oleh aturan yang dijalankan di Google SecOps.
rulesets: menyimpan informasi tentang deteksi pilihan Google SecOps,
termasuk kategori setiap set aturan, apakah diaktifkan, dan
status pemberitahuan saat ini.
udm_enum_value_to_name_mapping: Untuk jenis yang di-enum di tabel events, memetakan nilai numerik ke nilai string.
udm_events_aggregates: menyimpan data gabungan yang diringkas menurut jam
peristiwa yang dinormalisasi.
Mengakses data di BigQuery
Anda dapat menjalankan kueri secara langsung di BigQuery atau menghubungkan alat business intelligence Anda sendiri, seperti Looker atau Microsoft Power BI, ke BigQuery.
Untuk mengaktifkan akses ke instance BigQuery, gunakan
Google SecOps CLI atau
Google SecOps BigQuery Access API.
Anda dapat memberikan alamat email untuk pengguna atau grup yang Anda miliki. Jika Anda mengonfigurasi akses ke grup, gunakan grup tersebut untuk mengelola anggota tim yang dapat mengakses instance BigQuery.
Untuk menghubungkan Looker atau alat business intelligence lain ke BigQuery, hubungi perwakilan SecOps Google Anda untuk mendapatkan kredensial akun layanan yang memungkinkan Anda menghubungkan aplikasi ke set data BigQuery SecOps Google. Akun layanan akan memiliki peran IAM BigQuery Data Viewer (roles/bigquery.dataViewer) dan peran BigQuery Job Viewer (roles/bigquery.jobUser).
[[["Mudah dipahami","easyToUnderstand","thumb-up"],["Memecahkan masalah saya","solvedMyProblem","thumb-up"],["Lainnya","otherUp","thumb-up"]],[["Sulit dipahami","hardToUnderstand","thumb-down"],["Informasi atau kode contoh salah","incorrectInformationOrSampleCode","thumb-down"],["Informasi/contoh yang saya butuhkan tidak ada","missingTheInformationSamplesINeed","thumb-down"],["Masalah terjemahan","translationIssue","thumb-down"],["Lainnya","otherDown","thumb-down"]],["Terakhir diperbarui pada 2025-08-21 UTC."],[[["\u003cp\u003eGoogle Security Operations exports normalized and enriched telemetry data to BigQuery, enabling users to perform ad-hoc queries, use business intelligence tools, join with third-party datasets, and run advanced analytics.\u003c/p\u003e\n"],["\u003cp\u003eFrom December 31, 2024, only Enterprise Plus Tier customers will have access to the managed BigQuery data lake, with the managed resources and API keys fully deprecated by March 31, 2025, for others.\u003c/p\u003e\n"],["\u003cp\u003eThe exported data includes UDM event records, rule matches, IoC matches, ingestion metrics, and entity graph data, all stored in a customer-specific Google Cloud project managed by Google.\u003c/p\u003e\n"],["\u003cp\u003eData is exported on a fill-forward basis with a 365-day retention period, and raw logs are not exported to the Google Security Operations data lake in BigQuery.\u003c/p\u003e\n"],["\u003cp\u003eAccess to the BigQuery instance can be managed through the Google Security Operations CLI or API, and service account credentials for Looker and other BI tools can be obtained from a Google Security Operations representative.\u003c/p\u003e\n"]]],[],null,["# Google SecOps data in BigQuery\n==============================\n\nSupported in: \nGoogle secops [SIEM](/chronicle/docs/secops/google-secops-siem-toc)\n| **Note:** This option is available for Google SecOps Enterprise Plus customers only. For all other customers, see [Configure data export to BigQuery in a self-managed Google Cloud project](/chronicle/docs/preview/cloud-integration/export-to-customer-managed-project).\n\nGoogle SecOps provides a managed data lake of normalized and threat intelligence enriched\ntelemetry by exporting data to BigQuery. This lets you do the\nfollowing:\n\n- Run ad-hoc queries directly in BigQuery.\n- Use your own business intelligence tools, such as Looker or Microsoft Power BI, to create dashboards, reports, and analytics.\n- Join Google SecOps data with third-party datasets.\n- Run analytics using data science or machine learning tools.\n- Run reports using predefined default dashboards and custom dashboards.\n\nGoogle SecOps exports the following categories of data to BigQuery:\n\n- **UDM event records:** UDM records created from log data ingested by customers. These records are enriched with aliasing information.\n- **Rules matches (detections)**: instances where a rule matches one or more events.\n- **IoC matches**: artifacts (for example domains, IP addresses) from events that match Indicator of Compromise (IoC) feeds. This includes matches to from global feeds and customer-specific feeds.\n- **Ingestion metrics:** include statistics, such as number of log lines ingested, number of events produced from logs, number of log errors indicating that logs couldn't be parsed, and the state of Google SecOps forwarders. For more information, see [Ingestion metrics BigQuery schema](/chronicle/docs/reference/ingestion-metrics-schema).\n- **Entity graph and entity relationships**: stores the description of entities and their relationships with other entities.\n\nOverview of the tables\n----------------------\n\nGoogle SecOps creates the `datalake` dataset in BigQuery and the following tables:\n\n- `entity_enum_value_to_name_mapping`: for enumerated types in the `entity_graph` table, maps the numerical values to the string values.\n- `entity_graph`: stores data about UDM entities.\n- [`events`](/chronicle/docs/reports/events-schema-overview): stores data about UDM events.\n- [`ingestion_metrics`](/chronicle/docs/reference/ingestion-metrics-schema): stores statistics related to ingestion and normalization of data from specific ingestion sources, such as Google SecOps forwarders, feeds, and Ingestion API.\n- `ioc_matches`: stores IOC matches found against UDM events.\n- `job_metadata`: an internal table used to track the export of data to BigQuery.\n- `rule_detections`: stores detections returned by rules run in Google SecOps.\n- `rulesets`: stores information about Google SecOps curated detections, including the category each rule set belongs to, whether it is enabled, and the current alerting status.\n- `udm_enum_value_to_name_mapping`: For enumerated types in the events table, maps the numerical values to the string values.\n- `udm_events_aggregates`: stores aggregated data summarized by hour of normalized events.\n\nAccess data in BigQuery\n-----------------------\n\nYou can run queries directly in BigQuery or connect your own business\nintelligence tool, such as Looker or Microsoft Power BI, to BigQuery.\n\nTo enable access to the BigQuery instance, use the\n[Google SecOps BigQuery Access API](/chronicle/docs/reference/bigquery-access-api#access_api_reference).\nYou can provide an email address for either a user or a group that you own. If you\nconfigure access to a group, use the group to manage which team members can\naccess the BigQuery instance.\n\nTo connect Looker or another business intelligence tool to BigQuery, contact\nyour Google SecOps representative for service account credentials that enable you to\nconnect an application to the Google SecOps BigQuery dataset. The service\naccount will have IAM BigQuery Data Viewer role (`roles/bigquery.dataViewer`) and BigQuery Job Viewer role (`roles/bigquery.jobUser`).\n\nWhat's next\n-----------\n\n- Learn more about the following schemas:\n - [`events`](/chronicle/docs/reports/events-schema-overview)\n - [`ingestion_metrics`](/chronicle/docs/reference/ingestion-metrics-schema)\n- For information about accessing and running queries in BigQuery, see [Run interactive and batch query jobs](/bigquery/docs/running-queries).\n- For information about how to query partitioned tables, see [Query partitioned tables](/bigquery/docs/querying-partitioned-tables).\n- For information about how to connect Looker to BigQuery, see Looker documentation about [connecting to BigQuery](/looker/docs/db-config-google-bigquery).\n\n**Need more help?** [Get answers from Community members and Google SecOps professionals.](https://security.googlecloudcommunity.com/google-security-operations-2)"]]
