Collect Apache Tomcat logs

Supported in:

This document explains how to ingest Apache Tomcat logs to Google Security Operations using Bindplane. The parser extracts fields from JSON-formatted logs, transforming them into the Unified Data Model (UDM). It initializes default values, parses the JSON payload, handles potential JSON parsing errors, and maps various fields from the raw log into corresponding UDM fields, including metadata, principal, observer, and security result information, while also adding custom labels for environment context.

Before you begin

Make sure you have the following prerequisites:

  • Google SecOps instance
  • A Windows 2016 or later, or a Linux host with systemd
  • If running behind a proxy, firewall ports are open
  • Apache Tomcat version 9.0.70 or later
  • Write access to $CATALINA_BASE/conf and $CATALINA_BASE/logs

Get Google SecOps ingestion authentication file

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Collection Agents.
  3. Download the Ingestion Authentication File. Save the file securely on the system where Bindplane will be installed.

Get Google SecOps customer ID

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Profile.
  3. Copy and save the Customer ID from the Organization Details section.

Install the Bindplane agent on the Tomcat server to collect log files

Install the Bindplane agent on your Windows or Linux operating system according to the following instructions.

Windows installation

  1. Open the Command Prompt or PowerShell as an administrator.
  2. Run the following command:

    msiexec /i "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" /quiet
    

Linux installation

  1. Open a terminal with root or sudo privileges.
  2. Run the following command:

    sudo sh -c "$(curl -fsSlL https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh)" install_unix.sh
    

Additional installation resources

For additional installation options, consult the installation guide.

Configure the Bindplane agent to ingest Syslog and send to Google SecOps

  1. Access the configuration file:
    • Locate the config.yaml file. Typically, it's in the /etc/bindplane-agent/ directory on Linux or in the installation directory on Windows.
    • Open the file using a text editor (for example, nano, vi, or Notepad).
  2. Edit the config.yaml file as follows:

    receivers:
        filelog/tomcat:
        include: [ /path/to/tomcat/logs/access-log.*.json ]
        start_at: beginning
    
    exporters:
        chronicle/chronicle_w_labels:
            compression: gzip
            # Adjust the path to the credentials file you downloaded in Step 1
            creds_file_path: '/path/to/ingestion-authentication-file.json'
            # Replace with your actual customer ID from Step 2
            customer_id: <customer_id>
            endpoint: malachiteingestion-pa.googleapis.com
            # Add optional ingestion labels for better organization
            log_type: 'TOMCAT'
            raw_log_field: body
            ingestion_labels:
    
    service:
        pipelines:
            logs/source0__chronicle_w_labels-0:
                receivers:
                    - filelog/tomcat
                exporters:
                    - chronicle/chronicle_w_labels
    
    • Replace the /path/to/tomcat/logs.
    • Replace <customer_id> with the actual customer ID.
    • Update /path/to/ingestion-authentication-file.json to the path where the authentication file was saved in the Get Google SecOps ingestion authentication file section.

Restart the Bindplane agent to apply the changes

  • To restart the Bindplane agent in Linux, run the following command:

    sudo systemctl restart bindplane-agent
    
  • To restart the Bindplane agent in Windows, you can either use the Services console or enter the following command:

    net stop BindPlaneAgent && net start BindPlaneAgent
    

Configure JSON Access Logs in Tomcat

  1. Open the Tomcat file at $CATALINA_BASE/conf/server.xml.
  2. Locate the <Host> tag and add this inside:

    <Valve className="org.apache.catalina.valves.JsonAccessLogValve"
          directory="logs"
          prefix="access-log"
          suffix=".json"
          rotatable="true"
          maxDays="7"/>
    
  3. Restart Tomcat to apply changes:

    cd /path/to/tomcat
    bin/catalina.sh stop
    bin/catalina.sh start
    
  4. A new JSON log file will appear daily (for example, logs/access-log.2025-07-02.json).

UDM mapping table

Log Field UDM Mapping Logic
@timestamp metadata.event_timestamp The value of @timestamp from the raw log is directly mapped to this UDM field. It represents the time the event occurred.
agent.ephemeral_id additional.fields[ephemeral_id].value.string_value The ephemeral ID from the agent is added as a key-value pair in the additional fields.
agent.hostname observer.hostname The agent's hostname is used as the observer hostname.
agent.id observer.asset_id The agent ID is combined with the agent type to create the observer asset ID (e.g., filebeat: <agent_id>).
agent.type observer.application The agent type is used as the observer application.
agent.version observer.platform_version The agent version is used as the observer platform version.
host.hostname principal.hostname The host's hostname is used as the principal hostname.
host.id principal.asset.asset_id The host ID is prepended with Host Id: to create the principal asset ID.
host.ip principal.ip, observer.ip The host's IP address is used for both the principal and observer IP. If multiple IPs are present, they are merged into an array.
host.mac principal.mac The host's MAC address is used as the principal MAC address. If multiple MACs are present, they are merged into an array.
host.os.family principal.platform If the host OS family is rhel or redhat, the principal platform is set to LINUX.
host.os.kernel principal.platform_patch_level The host OS kernel version is used as the principal platform patch level.
host.os.name additional.fields[os_name].value.string_value The host OS name is added as a key-value pair in the additional fields.
host.os.version principal.platform_version The host OS version is used as the principal platform version.
log.file.path principal.process.file.full_path The log path is used as the principal process file's full path.
log_level security_result.severity, security_result.severity_details, security_result.action The log level is used to determine the security result severity, severity details, and action. DEBUG, INFO, and AUDIT map to INFORMATIONAL severity and ALLOW action. ERROR maps to ERROR severity and BLOCK action. WARNING and WARN map to MEDIUM severity and BLOCK action. The raw log_level value is also mapped to severity_details.
logstash.irm_environment additional.fields[irm_environment].value.string_value The Iron Mountain environment from Logstash is added as a key-value pair in the additional fields.
logstash.irm_region additional.fields[irm_region].value.string_value The Iron Mountain region from Logstash is added as a key-value pair in the additional fields.
logstash.irm_site additional.fields[irm_site].value.string_value The Iron Mountain site from Logstash is added as a key-value pair in the additional fields.
logstash.process.host intermediary.hostname The Logstash processing host is used as the intermediary hostname.
logstash.process.timestamp metadata.collected_timestamp The Logstash processing timestamp is used as the collected timestamp.
logstash.xyz_environment additional.fields[xyz_environment].value.string_value The xyz environment from Logstash is added as a key-value pair in the additional fields.
logstash.xyz_region additional.fields[xyz_region].value.string_value The xyz region from Logstash is added as a key-value pair in the additional fields.
logstash.xyz_site additional.fields[xyz_site].value.string_value The xyz site from Logstash is added as a key-value pair in the additional fields.
message metadata.description The message field is parsed as JSON and its event_message field is used as the metadata description. The intermediary application is hardcoded to logstash. The metadata event type is hardcoded to USER_UNCATEGORIZED. The metadata log type is set to TOMCAT from the raw log's batch.type or batch.log_type. The metadata product name is hardcoded to Tomcat. The metadata vendor name is hardcoded to Tomcat.
user principal.user.userid The user field from the raw log is used as the principal user ID.

Need more help? Get answers from Community members and Google SecOps professionals.