Collect VPC Flow Logs

This document explains how to export VPC Flow Logs to Google Security Operations using Google Cloud. The parser transforms the logs from their built-in JSON format into the Google Security Operations UDM. It extracts relevant fields like source and destination IP, port, protocol, and bytes sent, then maps them to corresponding UDM fields, taking into account network direction and special cases for accurate representation in Google SecOps.

Before You Begin

Ensure that you have the following prerequisites:

• Google SecOps instance.
• VPC Flow is set up and active in your Google Cloud environment.
• Privileged access to Google Cloud.

Create a Google Cloud Storage Bucket

1. Sign in to the Google Cloud console.

2. Go to the Cloud Storage Buckets page.

   Go to Buckets

3. Click Create.

4. On the Create a bucket page, enter your bucket information. After each of the following steps, click Continue to proceed to the next step:

   1. In the Get started section, do the following:

      1. Enter a unique name that meets the bucket name requirements; for example, vpcflow-logs.

      2. To enable hierarchical namespace, click the expander arrow to expand the Optimize for file oriented and data-intensive workloads section, and then select Enable Hierarchical namespace on this bucket.

      3. To add a bucket label, click the expander arrow to expand the Labels section.

      4. Click Add label, and specify a key and a value for your label.

   2. In the Choose where to store your data section, do the following:

      1. Select a Location type.

      2. Use the location type menu to select a Location where object data within your bucket will be permanently stored.

      3. To set up cross-bucket replication, expand the Set up cross-bucket replication section.

   3. In the Choose a storage class for your data section, either select a default storage class for the bucket, or select Autoclass for automatic storage class management of your bucket's data.

   4. In the Choose how to control access to objects section, select not to enforce public access prevention, and select an access control model for your bucket's objects.

   5. In the Choose how to protect object data section, do the following:

      1. Select any of the options under Data protection that you want to set for your bucket.
      2. To choose how your object data will be encrypted, click the expander arrow labeled Data encryption, and select a Data encryption method.

5. Click Create.

Configure Log Export in Google Cloud VPC Flow

1. Sign in to Google Cloud account using your privileged account.
2. On Welcome page, click VPC Networks.
3. Click Default and a subnet page should appear.
4. Select all logs.
5. Click Flow Logs > Configure.
6. Select Aggregation Interval; for example, 30 SEC.
7. Provide Sample Rate; for example, 50%.
8. Click Save
9. Search Logging in the search bar and click Enter.
10. In Log Explorer, filter the logs by choosing VPC_flows in the Log Name and click Apply.
11. Click More Actions.
12. Click Create Sink.
13. Provide the following configurations:
    1. Sink Details: enter a name and description.
    2. Click Next.
    3. Sink Destination: select Cloud Storage Bucket.
    4. Cloud Storage Bucket: select the bucket created earlier or create a new bucket.
    5. Click Next.
    6. Choose Logs to include in Sink: a default log is populated when you select an option in Cloud Storage Bucket.
    7. Click Next.
    8. Optional: Choose Logs to filter out of Sink: select the logs that you would like not to sink.
14. Click Create Sink.

Set up feeds

There are two different entry points to set up feeds in the Google SecOps platform:

• SIEM Settings > Feeds > Add New
• Content Hub > Content Packs > Get Started

How to set up the Google Cloud VPC Flow feed

1. Click the Google Cloud Compute platform pack.
2. Locate the GCP VPC Flow Feed logtype.

3. Specify the values in the following fields.

   • Source Type: Amazon SQS V2
   • Queue Name: The SQS queue name to read from
   • S3 URI: The bucket URI.
     • s3://your-log-bucket-name/
       • Replace your-log-bucket-name with the actual name of your S3 bucket.

   • Source deletion options: Select the deletion option according to your ingestion preferences.

   • Maximum File Age: Include files modified in the last number of days. Default is 180 days.

   • SQS Queue Access Key ID: An account access key that is a 20-character alphanumeric string.

   • SQS Queue Secret Access Key: An account access key that is a 40-character alphanumeric string.

   Advanced options

   • Feed Name: A prepopulated value that identifies the feed.
   • Asset Namespace: Namespace associated with the feed.
   • Ingestion Labels: Labels applied to all events from this feed.

4. Click Create feed.

For more information about configuring multiple feeds for different log types within this product family, see Configure feeds by product.

Supported VPC Flow Logs log formats

The VPC Flow Logs parser supports logs in JSON format.

Supported VPC Flow Logs Sample Logs

• JSON

  {
    "insertId": "1wjp1y9f8vc6y6",
    "jsonPayload": {
      "bytes_sent": "0",
      "connection": {
        "dest_ip": "198.51.100.0",
        "dest_port": 32846,
        "protocol": 6,
        "src_ip": "198.51.100.1",
        "src_port": 443
      },
      "dest_instance": {
        "project_id": "logging-259109",
        "region": "us-west2",
        "vm_name": "demisto-01",
        "zone": "us-west2-a"
      },
      "dest_vpc": {
        "project_id": "logging-259109",
        "subnetwork_name": "default",
        "vpc_name": "default"
      },
      "end_time": "2020-03-28T10:44:41.896734136Z",
      "packets_sent": "2",
      "reporter": "DEST",
      "start_time": "2020-03-28T10:44:41.896734136Z"
    },
    "logName": "projects/logging-259109/logs/compute.googleapis.com%2Fvpc_flows",
    "receiveTimestamp": "2020-03-28T10:44:50.112903743Z",
    "resource": {
      "labels": {
        "location": "us-west2-a",
        "project_id": "dummy_project_id",
        "subnetwork_id": "subnetwork_id",
        "subnetwork_name": "default"
      },
      "type": "gce_subnetwork"
    },
    "timestamp": "2020-03-28T10:44:50.112903743Z"
  }
  

UDM Mapping Table

Need more help? Get answers from Community members and Google SecOps professionals.