Collect HashiCorp audit logs

This parser processes HashiCorp audit logs in JSON, Syslog, or combined formats. It extracts fields, performs Grok and KV parsing for both standard and "runner" type messages, handles JSON payloads, and maps the extracted data to the UDM. The parser also includes error handling and dropping malformed logs.

Before you begin

• Ensure that you have a Google Security Operations instance.
• Ensure that you have a Windows 2016 or later or a Linux host with systemd.
• If running behind a proxy, ensure firewall ports are open.
• Ensure that you have privileged access to HCP.

Get Google SecOps ingestion authentication file

1. Sign in to the Google SecOps console.
2. Go to SIEM Settings > Collection Agents.
3. Download the Ingestion Authentication File.

Get Google SecOps customer ID

1. Sign in to the Google SecOps console.
2. Go to SIEM Settings > Profile.
3. Copy and save the Customer ID from the Organization Details section.

Install Bindplane Agent

1. For Windows installation, run the following script:
   msiexec /i "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" /quiet
2. For Linux installation, run the following script:
   sudo sh -c "$(curl -fsSlL https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh)" install_unix.sh
3. Additional installation options can be found in this installation guide.

Configure Bindplane Agent to ingest Syslog and send to Google SecOps

1. Access the machine where Bindplane Agent is installed.

2. Edit the config.yaml file as follows:

   receivers:
       udplog:
           # Replace the below port <54525> and IP <0.0.0.0> with your specific values
           listen_address: "0.0.0.0:54525" 
   
   exporters:
       chronicle/chronicle_w_labels:
           compression: gzip
           # Adjust the creds location below according the placement of the credentials file you downloaded
           creds: '{ json file for creds }'
           # Replace <customer_id> below with your actual ID that you copied
           customer_id: <customer_id>
           endpoint: malachiteingestion-pa.googleapis.com
           # You can apply ingestion labels below as preferred
           ingestion_labels:
           log_type: SYSLOG
           namespace: auditd
           raw_log_field: body
   service:
       pipelines:
           logs/source0__chronicle_w_labels-0:
               receivers:
                   - udplog
               exporters:
                   - chronicle/chronicle_w_labels
   

3. Restart the Bindplane Agent to apply the changes:

   sudo systemctl restart bindplane
   

Enable Syslog for HCP Vault

1. Sign in to the HCP Portal.
2. Go to Vault Clusters.
3. Select your Vault cluster from the list of deployed clusters.
4. In the Cluster Overview, locate and copy the Vault Address (for example, https://vault-cluster-name.hashicorpcloud.com:8200).
5. Go to the Access Details section, and copy the Root Token.

Install the Vault CLI

• For Linux:

  curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
  echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
  sudo apt update && sudo apt install vault
  

• For macOS (using Homebrew):

  brew tap hashicorp/tap
  brew install hashicorp/tap/vault
  

• For Windows:

  Download the executable file.
  Extract it and add the Vault binary to your system's PATH.
  

• Verify the Vault CLI installation by running:

  vault --version
  

Configure HCP Vault using CLI to send audit logs to Bindplane

1. Open the terminal or the command prompt.

2. Set the Vault server address using the environment variable:

   export VAULT_ADDR="https://vault-cluster-name.hashicorpcloud.com:8200"
   

3. Log in to Vault using the root token:

   vault login <root-token>
   

Configure the Syslog Path to an External Syslog Socket

1. Run the following command to enable syslog and send to Bindplane Agent:

   vault audit enable socket address="udp://<bindplane-ip>:<bindplane-port>" socket_type="udp" tag="vault"
   

2. Confirm the new configuration:

   vault audit list
   

3. The output should display the new socket configuration.

4. Optional: Automate the setup using Terraform:

   • Create a Terraform configuration file (audit.tf):

   resource "vault_audit" "syslog" {
     type        = "syslog"
     description = "Syslog audit logs"
     options = {
       tag      = "vault"
       facility = "LOCAL0"
     }
   }
   
   resource "vault_audit" "socket" {
     type        = "socket"
     description = "Remote syslog socket"
     options = {
       address     = "udp://<syslog-server-ip>:514"
       socket_type = "udp"
       tag         = "vault"
     }
   }
   

   • Apply the configuration:

   terraform init
   terraform apply
   

Troubleshooting logs not received

• Verify the syslog server is reachable:

  ping <syslog-server-ip>
  

UDM Mapping Table

Need more help? Get answers from Community members and Google SecOps professionals.