Collect Nix System Red Hat logs

This document explains how to ingest RHEL Server (Unix System) logs to Google Security Operations using Bindplane. The parser ingests syslog and JSON formatted logs, initializes a wide array of UDM fields to empty strings, performs several string substitutions on the message field, and then attempts to parse the message as JSON. If JSON parsing fails, it uses grok patterns to extract fields based on the message and event_details.original content, mapping the extracted fields to the UDM based on the event type and various conditional checks, handling different log formats and structures from various Unix system processes and services.

Before you begin

Ensure that you have the following prerequisites:

• Google SecOps instance
• If running behind a proxy, firewall ports are open
• Privileged access to a RHEL server

Get Google SecOps ingestion authentication file

1. Sign in to the Google SecOps console.
2. Go to SIEM Settings > Collection Agents.
3. Download the Ingestion Authentication File. Save the file securely on the system where Bindplane will be installed.

Get Google SecOps customer ID

1. Sign in to the Google SecOps console.
2. Go to SIEM Settings > Profile.
3. Copy and save the Customer ID from the Organization Details section.

Install the Bindplane agent

Linux installation

1. Open a terminal with root or sudo privileges.

2. Run the following command:

   sudo sh -c `$(curl -fsSlL https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh)` install_unix.sh
   

Additional installation resources

For additional installation options, consult the installation guide.

Configure the Bindplane agent to ingest Syslog and send to Google SecOps

1. Access the configuration file:
   • Locate the config.yaml file. Typically, it's in the /etc/bindplane-agent/ directory on Linux.
   • Open the file using a text editor (for example, nano or vi).

2. Edit the config.yaml file as follows:

         receivers:
             filelog/linux:
             include:
               - /var/log/messages
               - /var/log/lastlog
               - /var/log/btmp
               - /var/log/wtmp
               - /var/log/secure
               - /var/log/cron
               - /var/log/maillog
               - /var/log/boot
             start_at: end
             poll_interval: 5s
   
         exporters:
             chronicle/linux:
                 # Adjust the path to the credentials file you downloaded in Step 1
                 creds: '/path/to/ingestion-authentication-file.json'
                 # Replace with your actual customer ID from Step 2
                 customer_id: <customer_id>
                 endpoint: malachiteingestion-pa.googleapis.com
                 # Add optional ingestion labels for better organization
                 log_type: 'NIX_SYSTEM'
                 override_log_type: false
                 raw_log_field: body
   
         service:
           pipelines:
             logs/linux:
               receivers:
               - filelog/linux
           exporters: [chronicle/linux]
         ```
   

1. Replace <customer_id> with the actual customer ID.

2. Update /path/to/ingestion-authentication-file.json to the path where the authentication file was saved in the Get Google SecOps ingestion authentication file section.

   Start the Bindplane agent and apply changes

3. Start the Bindplane agent:

   sudo systemctl start bindplane-agent
   

4. Enable the observIQ otel collector Service:

   systemctl enable --now bindplane-agent
   

5. Restart the Bindplane agent if needed:

   sudo systemctl restart bindplane-agent
   

UDM mapping table

• event.idm.read_only_udm.metadata.event_type: Set to GENERIC_EVENT initially, then overwritten based on parser logic.
• event.idm.read_only_udm.metadata.product_name: Hardcoded to Unix System.
• event.idm.read_only_udm.extensions.auth.type: Set to MACHINE for certain event types.
• event.idm.read_only_udm.target.asset.attribute.cloud.environment: Set to GOOGLE_CLOUD_PLATFORM for Google Cloud audit logs.
• event.idm.read_only_udm.target.resource.resource_type: Set to VIRTUAL_MACHINE for Google Cloud audit logs.
• event.idm.read_only_udm.extensions.auth.mechanism: Set to USERNAME_PASSWORD for login events.
• has_target_resource: Set to true if resource.labels.instance_id or _Internal_WorkspaceResourceId is present.

Need more help? Get answers from Community members and Google SecOps professionals.