Collect CircleCI audit logs

This parser extracts fields from CircleCI audit logs in CSV and JSON formats, transforming them into the Unified Data Model (UDM). It handles both formats, performs data transformations and enrichments, and maps the extracted fields to their corresponding UDM fields within the event object. It focuses on user actions, resource access, and update events, categorizing them and populating relevant UDM fields like principal, target, network, and metadata.

Before you begin

• Ensure that you have a Google SecOps instance.
• Ensure that you have privileged access to CircleCI.

Configure a feed in Google SecOps to ingest the CircleCI logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, CircleCI Logs).
4. Select Webhook as the Source type.
5. Select CircleCI as the Log type.
6. Click Next.
7. Optional: Specify values for the following input parameters:
   • Split delimiter: the delimiter that is used to separate log lines, such as \n.
   • Asset namespace: the asset namespace.
   • Ingestion labels: the label applied to the events from this feed.
8. Click Next.
9. Review the feed configuration in the Finalize screen, and then click Submit.
10. Click Generate Secret Key to generate a secret key to authenticate this feed.
11. Copy and store the secret key. You cannot view this secret key again. If needed, you can regenerate a new secret key, but this action makes the previous secret key obsolete.
12. From the Details tab, copy the feed endpoint URL from the Endpoint Information field. You need to specify this endpoint URL in your client application.
13. Click Done.

Create an API key for the webhook feed

1. Go to Google Cloud console > Credentials.

   Go to Credentials

2. Click Create credentials, and then select API key.

3. Restrict the API key access to the Google Security Operations API.

Specify the endpoint URL

1. In your client application, specify the HTTPS endpoint URL provided in the webhook feed.

2. Enable authentication by specifying the API key and secret key as part of the custom header in the following format:

   X-goog-api-key = API_KEY
   X-Webhook-Access-Key = SECRET
   

   Recommendation: Specify the API key as a header instead of specifying it in the URL.

3. If your webhook client doesn't support custom headers, you can specify the API key and secret key using query parameters in the following format:

   ENDPOINT_URL?key=API_KEY&secret=SECRET
   

   Replace the following:

   • ENDPOINT_URL: the feed endpoint URL.
   • API_KEY: the API key to authenticate to Google SecOps.
   • SECRET: the secret key that you generated to authenticate the feed.

Configuring a webhook in CircleCI

1. Sign in to the CircleCI web interface.
2. Select the project you want to ingest the logs from.
3. Click Project Settings.
4. Select Webhooks.
5. Click Add Webhook.

6. Specify values for the following input parameters:

   • Webhook Name: provide a descriptive name (for example, Google SecOps).
   • Endpoint URL: enter the <ENDPOINT_URL> of the Google SecOps API endpoint.
   • Events: Select the CircleCI events that should trigger the webhook (for example, select workflow-completed to send data after a workflow finishes).

7. Click Save to create the webhook.

UDM Mapping Table

Changes

2023-03-09

• Newly created parser.