Collect F5 BIG-IP ASM logs

Supported in:

This document explains how to ingest F5 BIG-IP Application Security Manager (ASM) logs to Google Security Operations by using Bindplane. The parser handles various log formats (syslog, CSV, CEF, etc.) and normalizes them into UDM. It uses grok patterns and key-value extractions to parse fields, XML filtering for violation details, conditional logic for event categorization and severity mapping, and merges extracted fields into the UDM schema.

Before you begin

  • Ensure that you have a Google Security Operations instance.
  • Ensure that you are using Windows 2016 or later, or a Linux host with systemd.
  • If running behind a proxy, ensure firewall ports are open.
  • Ensure that you have privileged access to F5 BIG-IP ASM.

Get Google SecOps ingestion authentication file

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Collection Agents.
  3. Download the Ingestion Authentication File. Save the file securely on the system where Bindplane will be installed.

Get Google SecOps customer ID

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Profile.
  3. Copy and save the Customer ID from the Organization Details section.

Install the Bindplane agent

Windows installation

  1. Open the Command Prompt or PowerShell as an administrator.

  2. Run the following command:

    msiexec /i "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" /quiet

Linux installation

  1. Open a terminal with root or sudo privileges.

  2. Run the following command:

    sudo sh -c "$(curl -fsSlL https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh)" install_unix.sh

Additional installation resources

Configure the Bindplane agent to ingest Syslog and send to Google SecOps

  1. Access the configuration file:

    1. Locate the config.yaml file. Typically, it's in the /etc/bindplane-agent/ directory on Linux or in the installation directory on Windows.
    2. Open the file using a text editor (for example, nano, vi, or Notepad).

  2. Edit the config.yaml file as follows:

    receivers:
    udplog:
        # Replace the port and IP address as required
        listen_address: "0.0.0.0:514"

exporters:
    chronicle/chronicle_w_labels:
        compression: gzip
        # Adjust the path to the credentials file you downloaded in Step 1
        creds: '/path/to/ingestion-authentication-file.json'
        # Replace with your actual customer ID from Step 2
        customer_id: <customer_id>
        endpoint: malachiteingestion-pa.googleapis.com
        # Add optional ingestion labels for better organization
        ingestion_labels:
            log_type: F5_ASM
            raw_log_field: body

service:
    pipelines:
        logs/source0__chronicle_w_labels-0:
            receivers:
                - udplog
            exporters:
                - chronicle/chronicle_w_labels

  3. Replace the port and IP address as required in your infrastructure.

  4. Replace <customer_id> with the actual customer ID.

  5. Update /path/to/ingestion-authentication-file.json to the path where the authentication file was saved in the Get Google SecOps ingestion authentication file section.

Restart the Bindplane agent to apply the changes

  • To restart the Bindplane agent in Linux, run the following command:

    sudo systemctl restart bindplane-agent

  • To restart the Bindplane agent in Windows, you can either use the Services console or enter the following command:

    net stop BindPlaneAgent && net start BindPlaneAgent

Configure Remote Logging on F5 BIG-IP ASM

  1. Sign in to the ASM Console Web UI.
  2. Go to Security > Event Logs > Logging Profiles.
  3. Click Create.

  4. Provide the following configuration details:

    • Profile Name: enter a unique name for the profile.
    • Select Application Security.
    • On the Application Security tab, select Advanced (if additional configurations are required).
    • Storage Destination: select Remote Storage.
    • Logging Format: Select Common Event Format (CEF).
    • Clear Local storage.
    • Protocol: select UDP or TCP (depending on your Bindplane agent configuration).
    • Server Addresses: enter the Bindplane agent IP Address.
    • Port: selected by default 514. Update the port setting according to your Bindplane agent configuration.
    • Click Add.
    • Select Guarantee logging.
    • Select Report detected anomalies.
    • Facility: select LOG_LOCAL6 (you can optionally select the facility category of the logged traffic, the possible values are LOG_LOCAL0 through LOG_LOCAL7).

  5. Click Finished.

Associate a logging profile with a security policy

  1. Click Local Traffic > Virtual Servers.
  2. Click the name of the virtual server used by the security policy.
  3. From the Security menu, select Policies.
  4. Ensure that the Application Security Policy setting is Enabled, and that Policy is set to the security policy you want.
  5. Ensure that the Log Profile setting is set to Enabled.
  6. From the Available list, select the profile to use for the security policy, and move it into the Selected list.
  7. Click Update.

UDM Mapping Table

Log Field UDM Mapping Logic
act security_result.action If act is blocked, maps to BLOCK. If act is passed or legal, maps to ALLOW. If act contains alerted, maps to QUARANTINE. Otherwise, defaults to ALLOW for Splunk format.
app network.application_protocol Directly maps to HTTPS if present in the raw log.
attack_type security_result.category_details, metadata.description Used in conjunction with other fields to determine security_result.category. If no other description is available, it becomes the event description. For Splunk format logs, it is used to determine category and summary if violations is empty.
client_ip principal.ip, principal.asset.ip Directly maps to principal IP.
cn1 network.http.response_code Directly maps to HTTP response code.
cn2 security_result.severity_details Directly maps to security result severity details. Used with response_code to determine if an event is an alert.
column1 principal.ip, principal.asset.ip Maps to principal IP for certain CSV formatted logs.
column2 target.port Maps to target port for certain CSV formatted logs.
column3 target.ip, target.asset.ip Maps to target IP for certain CSV formatted logs.
column4 security_result.severity Maps to security result severity for certain CSV formatted logs. Values Information, Informational, 0, 4 map to INFORMATIONAL. Warning, 1, 3 map to MEDIUM. Error, 2 map to ERROR. Critical, CRITICAL, critical map to CRITICAL.
column7 security_result.detection_fields, network.http.response_code Contains XML data. viol_name within request-violations is extracted and added as detection fields with key Request Violation Name_index. viol_name within response_violations is extracted and added as detection fields with key Response Violation Name_index. response_code within response_violations maps to network.http.response_code.
column8 security_result.rule_name Maps to security result rule name for certain CSV formatted logs.
cs1 security_result.rule_name Directly maps to security result rule name.
cs2 security_result.summary Directly maps to security result summary.
cs5 principal.ip, principal.asset.ip, additional.fields If cs5 contains a JNDI LDAP URL, it is added as an additional field with key JNDI_LDAP_URL. Otherwise, if it contains comma-separated IPs, any IP different from principal_ip is added as an additional principal IP.
cs6 principal.location.country_or_region Directly maps to principal location country or region.
data network.session_id, network.sent_bytes, network.tls.version If present, parsed as JSON to extract sessionid, bits (mapped to sent_bytes), and version.
date_time metadata.event_timestamp Directly maps to event timestamp after parsing and converting to the correct format.
dest_ip target.ip, target.asset.ip Directly maps to target IP.
dest_port target.port Directly maps to target port.
dhost target.hostname Directly maps to target hostname.
dpt target.port Directly maps to target port.
dst target.ip Directly maps to target IP.
dvc intermediary.ip Directly maps to intermediary IP.
dvchost target.hostname, intermediary.hostname Directly maps to target hostname and intermediary hostname.
errdefs_msgno additional.fields Added as an additional field with key errdefs_msgno.
externalId additional.fields Added as an additional field with key Support_Id.
f5_host target.hostname, intermediary.hostname Directly maps to target hostname and intermediary hostname.
geo_info principal.location.country_or_region, security_result.detection_fields Maps to principal location country or region. Also added as a detection field with key geo_info.
host target.hostname Directly maps to target hostname.
ids additional.fields Parsed as a comma-separated list of support IDs. Each ID is added to a list-valued additional field with key supportid.
ip_addr_intelli security_result.detection_fields Added as a detection field with key ip_addr_intelli.
ip_client principal.ip Directly maps to principal IP.
ip_route_domain principal.ip, principal.asset.ip The IP portion is extracted and mapped to principal IP.
irule security_result.rule_name Directly maps to security result rule name.
irule-version security_result.rule_version Directly maps to security result rule version.
level security_result.severity, security_result.severity_details Used to determine security result severity. error or warning map to HIGH. notice maps to MEDIUM. information or info map to LOW. The raw value is also mapped to severity_details.
logtime metadata.event_timestamp Directly maps to event timestamp after parsing.
management_ip_address, management_ip_address_2 intermediary.ip Directly maps to intermediary IP.
method network.http.method Directly maps to HTTP method.
msg security_result.summary, metadata.description Directly maps to security result summary for some log formats. If no other description is available, it becomes the event description.
policy_name security_result.about.resource.name, security_result.rule_name Directly maps to security result resource name or rule name.
process target.application Directly maps to target application.
process_id principal.process.pid Directly maps to principal process ID.
protocol network.application_protocol, network.ip_protocol, app_protocol Directly maps to application protocol or IP protocol depending on the log format.
proxy_id security_result.rule_id Directly maps to security result rule ID.
query_string additional.fields Added as an additional field with key query_string.
referrer network.http.referral_url Directly maps to HTTP referral URL.
req_method network.http.method Directly maps to HTTP method.
req_status security_result.action, security_result.action_details, security_result.detection_fields If blocked, maps security_result.action to BLOCK. If passed or legal, maps to ALLOW. If contains alerted, maps to QUARANTINE. The raw value is also mapped to action_details and added as a detection field with key req_status.
request target.url Directly maps to target URL.
requestMethod network.http.method Directly maps to HTTP method.
resp security_result.detection_fields Added as a detection field with key resp.
resp_code network.http.response_code Directly maps to HTTP response code.
response security_result.summary Directly maps to security result summary.
response_code network.http.response_code Directly maps to HTTP response code.
route_domain additional.fields Added as an additional field with key route_domain.
rt metadata.event_timestamp Directly maps to event timestamp after parsing.
sev security_result.severity, security_result.severity_details Used to determine security result severity. ERROR maps to ERROR. The raw value is also mapped to severity_details.
severity security_result.severity, security_result.severity_details Used to determine security result severity. Informational maps to LOW, Error or warning map to HIGH, critical maps to CRITICAL, notice maps to MEDIUM, information or info map to LOW. The raw value is also mapped to severity_details.
sig_ids security_result.rule_id Directly maps to security result rule ID.
sig_names security_result.rule_name Directly maps to security result rule name.
snat_ip principal.nat_ip Directly maps to principal NAT IP.
snat_port principal.nat_port Directly maps to principal NAT port.
src principal.ip, principal.asset.ip Directly maps to principal IP.
spt principal.port Directly maps to principal port.
sub_violates security_result.about.resource.attribute.labels Added as a label with key Sub Violations to security result resource attributes.
sub_violations security_result.about.resource.attribute.labels Added as a label with key Sub Violations to security result resource attributes.
summary security_result.summary Directly maps to security result summary.
support_id metadata.product_log_id Prefixed with support_id - and mapped to product log ID.
suid network.session_id Directly maps to network session ID.
suser principal.user.userid Directly maps to principal user ID.
timestamp metadata.event_timestamp Directly maps to event timestamp after parsing and converting to the correct format.
unit_host principal.hostname, principal.asset.hostname Directly maps to principal hostname.
uri principal.url Directly maps to principal URL.
user_id principal.user.userid Directly maps to principal user ID.
user_name principal.user.user_display_name Directly maps to principal user display name.
username principal.user.userid Directly maps to principal user ID.
useragent network.http.user_agent, network.http.parsed_user_agent Directly maps to HTTP user agent. Also parsed and mapped to parsed user agent.
virtualserver network.tls.client.server_name Directly maps to TLS client server name.
violate_details security_result.detection_fields, network.http.response_code Contains XML data. viol_name within request-violations is extracted and added as detection fields with key Request Violation Name_index. viol_name within response_violations is extracted and added as detection fields with key Response Violation Name_index. response_code within response_violations maps to network.http.response_code.
violate_rate security_result.detection_fields Added as a detection field with key violate_rate.
violation_rating security_result.about.resource.attribute.labels Added as a label with key Violations Rating to security result resource attributes.
violations security_result.description Directly maps to security result description. For Splunk format logs, it is used to determine summary if present.
virus_name security_result.threat_name Directly maps to security result threat name.
vs_name network.tls.client.server_name Directly maps to TLS client server name.
websocket_direction network.direction If clientToServer, maps to INBOUND. If ServerToclient, maps to OUTBOUND.
websocket_message_type security_result.detection_fields Added as a detection field with key WebsocketMessageType.
x_fwd_hdr_val principal.ip, principal.asset.ip Directly maps to principal IP.

Changes

2025-02-11

Enhancement:

  • Mapped column3 to principal.ip and principal.asset.ip

2025-02-04

Enhancement:

  • Added a gsub to remove non-utf8 characters from uri field when it contains non-utf8 characters to parse logs.

2025-01-30

Enhancement:

  • Removed the cs5 field from _intermediary.ip and _intermediary.asset.ip.
  • Mapped src to principal.nat_ip.
  • Mapped cs5 to principal.ip and principal.asset.ip.

2025-01-17

Enhancement:

  • Removed the drop condition to parse logs with non-utf8 characters.

2024-12-11

Enhancement:

  • Modified a Grok Pattern to support a new format of syslog logs.

2024-11-28

Enhancement:

  • Changed the mapping of Referer from network.http.referral_url to target.url.

2024-11-07

Enhancement:

  • Mapped exec_data to target.process.command_line.
  • Mapped src to principal.hostname and principal.asset.hostname.
  • Mapped cs3 to additional.fields.

2024-10-30

Enhancement:

  • Added support to handle CSV logs.

2024-10-28

Enhancement:

  • Modified existing Grok pattern to handle ISP block and ISP GEO block.

2024-10-25

Enhancement:

  • Mapped form_data to additional.fields.

2024-10-23

Enhancement:

  • Mapped SOAPAction to additional.fields.

2024-09-30

Enhancement:

  • Mapped link to target.url
  • When the message contains DROP then set security_result.action to BLOCK.
  • When the message contains allowed then set security_result.action to ALLOW.

2024-08-07

Enhancement:

  • Modified existing Grok pattern to handle CEF logs.
  • Mapped suid to principal.user.userid.
  • Mapped suser to principal.user.user_display_name.
  • Mapped device_version to metadata.product_version.
  • Mapped severity to security_result.severity.

2024-07-15

Enhancement:

  • Added support to handle the SYSLOG + KV logs.

2024-06-17

Enhancement:

  • Added support for a new pattern of CSV logs.

2024-06-11

Enhancement:

  • Added KV block to handle unparsed KV logs.
  • Formatted CSV logs using gsub to parse CSV logs.

2024-05-13

Enhancement:

  • Added KV block to parse KV logs.
  • Added gsub to remove unwanted characters.

2024-04-19

Enhancement:

  • Handled CSV unparsed logs.
  • Added a Grok pattern to map resp_code.
  • Mapped errdefs_msgno, support_id_array, audit_component to additional.fields.
  • Mapped descrip to metadata.description.

2024-04-08

Enhancement:

  • Added support to parse newly ingested unparsed logs.

2024-04-05

Bug fix:

  • Added condition to parse dropped ASF CEM logs.

2024-02-27

Bug fix:

  • When cs5 field has a valid IP address, then mapped to principal.ip.
  • Aligned principal.ip and principal.asset.ip mappings.
  • Aligned principal.hostname and principal.asset.hostname mappings.
  • Aligned target.ip and target.asset.ip mappings.
  • Aligned target.hostname and target.asset.hostname mappings.

2024-01-12

Enhancement:

  • Mapped severity to security_result.severity_details.
  • Mapped resp_code to http.response_code.
  • Mapped virus_name to security_result.threat_name.
  • Mapped ip_route_domain to principal.ip.
  • Mapped geo_info, resp, req_status, violate_rate, and ip_addr_intelli to security_result.detection_fields.

2023-12-15

Enhancement:

  • Handled newly ingested set of logs where metadata.event_type is GENERIC_EVENT and network.application_protocol is HTTP.
  • Set network.ip_protocol to UDP if message contains UDP.
  • Removed hardcoding value of network.application_protocol.
  • Set network.application_protocol to HTTP and HTTPS if message has HTTP and `HTTPS, respectively.
  • Set network.application_protocol to HTTP if metadata.event_type is NETWORK_HTTP.
  • Added two Grok patterns to parse principal_ip and src_port from newly ingested logs.
  • Mapped message_body to metadata.description.
  • Mapped tmm_msg to metadata.description

2023-12-07

Enhancement:

  • Added a new Grok pattern to parse new KV+XML logs.
  • Added KV filters to parse unparsed KV logs.
  • Added XML filters to parse unparsed XML logs.
  • Mapped policy_name to security_result.about.resource.name.
  • Mapped viol_name to security_result.detection_fields.
  • Mapped response_code to network.http.response_code.
  • Modified Grok pattern to map complete Referer field to network.http.referral_url.
  • Mapped parseduseragent to `network.http.parsed_user_agent.

2023-11-08

Enhancement:

  • Added a new Grok pattern to parse new KV logs.
  • Added a KV filter to parse uparsed KV logs.
  • Mapped bigip_mgmt_ip, client_ip_geo_location, client_port, client_request_uri, device_version, http_method, route_domain and virtual_server_name to principal.ip, principal.location.country_or_region, principal.port, principal.url, metadata.product_version, network.http.method, additional.fields, network.tls.client.server_name, respectively.
  • Added legal to request_status condition to map security_result.action_details as ALLOW.
  • Mapped profile_name, action, previous_action, bot_signature, bot_signature_category, bot_name, class, anomaly_categories, anomalies, micro_services_name, micro_services_type, micro_services_matched_wildcard_url, micro_services_hostname, browser_configured_verification_action, browser_actual_verification_action, new_request_status, mobile_is_app, enforced_by, application_display_name, client_type, and challenge_failure_reason to additional.fields.

2023-10-19

Enhancement:

  • Added a Grok pattern to extract the value of Referer field as referer from CEF logs.
  • Mapped referer to network.http.referral_url.

2023-09-27

Bug fix:

  • Set security_result.action to BLOCK and security_result.action_details to blocked for logs having request_status = blocked.
  • Set security_result.action to ALLOW and security_result.action_details to passed for logs having request_status = passed.
  • Set security_result.action to QUARANTINE and security_result.action_details to alerted for logs having request_status = alerted.

2023-08-07

Enhancement:

  • Mapped management_ip_address to metadata.intermediary.ip.
  • Mapped request_status to security_result.action.
  • Mapped query_string to additional.fields.
  • Mapped sig_ids to security_result.rule_id.
  • Mapped sig_names to security_result.rule_name.
  • Mapped username to principal.user.userid.
  • Mapped policy_name to security_result.about.resource.name.
  • Mapped sub_violations to security_result.about.resource.attribute.labels.
  • Mapped violation_rating to security_result.about.resource.attribute.labels.
  • Mapped websocket_direction to network.direction.
  • Mapped websocket_message_type to security_result.detection_fields.

2023-07-27

Bug fix:

  • Added a new field target_app to contain value corresponding to target.application.
  • Mapped the field process to target.application only when value of the field target_app is null.
  • Converted the field process to string if it's already not a string.

2023-07-03

Enhancement:

  • Mapped externalId to `additional.fields.
  • Mapped the event time to `metadata.event_timestamp.

2023-05-12

Enhancement:

  • For CEF format logs, mapped the information about the attack to security_result.description.

2023-04-06

Enhancement:

  • Login event parsed as 'USER_LOGIN' instead of 'STATUS UPDATE'.
  • Parsed the username value in 'firstname.lastname' and mapped to 'principal.user.userid'.

2023-02-09

Enhancement:

  • Parsed the logs containing type=irule by adding new grok pattern and mapped the following fields:
  • Mapped type to metadata.product_event_type.
  • Mapped data.sessionid to network.session_id.
  • Mapped data.bits to network.sent_bytes.
  • Mapped data.version to network.tls.version.
  • Mapped client_ip to principal.ip.
  • Mapped client_port to principal.port.
  • Mapped snat_ip to principal.nat_ip.
  • Mapped snat_port to principal.nat_port.
  • Mapped server_ip to target.ip.
  • Mapped server_port to target.port.
  • Mapped irule to security_result.rule_name.
  • Mapped irule-version to security_result.rule_version.
  • Mapped proxy_id to security_result.rule_id.
  • Mapped virtualserver to network.tls.client.server_name.

2022-11-03

Enhancement:

  • Added a condition for unparsed CEF format logs.
  • Added a condition to check for sshd and httpd user_login logs.
  • Added grok patterns to parse httpd and sshd user_login success/failure logs.
  • Mapped event_id to metadata.product_log_id.
  • Mapped application to target.application.
  • Mapped prin_ip to principal.ip.
  • Mapped SSH to app_protocol when tty is ssh or applicaition is sshd.
  • Mapped user_id principal.user.user_id.
  • Mapped USER_LOGIN to metadata.event_type for httpd/sshd user_login logs.
  • Mapped auth_level to principal.user.attribute.roles.
  • Mapped addr from log to target.ip
  • Mapped port from log to target.port

2022-09-21

Enhancement:

  • Migrated customer specific to default parser.

2022-05-17

Enhancement:

  • Enhanced the parser to parse the header of the HTTP request.

2022-04-27

Bug fix:

  • Enhanced the parser to parse logs with the ASM: format.

2022-04-26

Enhancement:

  • Enhanced the parser to handle unparsed raw logs

Need more help? Get answers from Community members and Google SecOps professionals.