Change log for F5_ASM
| Date | Changes |
|---|---|
| 2025-08-20 | Enhancement:
- event.idm.read_only_udm.principal.ip: Newly mapped `net.host.ip` raw log field(s) with `event.idm.read_only_udm.principal.ip` UDM field. - event.idm.read_only_udm.principal.asset.ip: Newly mapped `net.host.ip` raw log field(s) with `event.idm.read_only_udm.principal.asset.ip` UDM field. - event.idm.read_only_udm.target.ip: Newly mapped `net.peer.ip` raw log field(s) with `event.idm.read_only_udm.target.ip` UDM field. - event.idm.read_only_udm.target.asset.ip: Newly mapped `net.peer.ip` raw log field(s) with `event.idm.read_only_udm.target.asset.ip` UDM field. - event.idm.read_only_udm.target.hostname: Newly mapped `net.peer.name` raw log field(s) with `event.idm.read_only_udm.target.hostname` UDM field. - event.idm.read_only_udm.target.asset.hostname: Newly mapped `net.peer.name` raw log field(s) with `event.idm.read_only_udm.target.asset.hostname` UDM field. - event.idm.read_only_udm.principal.hostname: Newly mapped `net.host.name` raw log field(s) with `event.idm.read_only_udm.principal.hostname` UDM field. - event.idm.read_only_udm.principal.asset.hostname: Newly mapped `net.host.name` raw log field(s) with `event.idm.read_only_udm.principal.asset.hostname` UDM field. - event.idm.read_only_udm.principal.port: Newly mapped `net.host.port` raw log field(s) with `event.idm.read_only_udm.principal.port` UDM field. - event.idm.read_only_udm.target.port: Newly mapped `net.peer.port` raw log field(s) with `event.idm.read_only_udm.target.port` UDM field. - event.idm.read_only_udm.target.resource.name: Newly mapped `vs_name` raw log field(s) with `event.idm.read_only_udm.target.resource.name` UDM field. - event.idm.read_only_udm.additional.fields: Newly mapped `captcha_result`, `policy_apply_date` and `request` raw log field(s) with `event.idm.read_only_udm.additional.fields` UDM field. - Converted type to boolean with error handling for `message_json_parse_failure_conversion_error`. - Modified the drop logic to only drop if the message is not JSON and doesn't match other patterns. |
| 2025-08-11 | Enhancement:
- Added JSON filter to parse unparsed JSON logs. - Consolidated all mapping for event.idm.read_only_udm.additional.fields, event.idm.read_only_udm.security_result.detection_fields. |
| 2025-07-10 | Enhancement:
- Added grok patterns to extract `action` raw log field from the logs. - event.idm.read_only_udm.security_detection_fields: Newly mapped `action`, `dos_attack_detection_mode`, `dos_attack_event`, `dos_attack_id`, `dos_attack_latency`, `dos_attack_name`, `dos_attack_tps`, `dos_mitigation_action`, and `dos_mitigation_reason` raw logs field with `event.idm.read_only_udm.security_detection_fields` UDM field. - event.idm.read_only_udm.additional.fields: Newly mapped 'context_type', 'dos_baseline_latency', `dos_baseline_tps`, 'dos_incoming_requests_count', 'dos_dropped_requests_count', 'errdefs_msg_name', `reported_entity_type` and `event_id` raw log fields with `event.idm.read_only_udm.additional.fields` UDM field. - event.idm.read_only_udm.principal.ip & event.idm.read_only_udm.principal.asset.ip: Newly mapped `source_ip` raw log field with `event.idm.read_only_udm.principal.ip` and `event.idm.read_only_udm.principal.asset.ip` UDM field and set `has_principal` to `true`. - event.idm.read_only_udm.intermediary.asset.attribute.labels: Newly mapped `device_blade` and `partition_name` raw log fields with `event.idm.read_only_udm.intermediary.asset.attribute.labels` UDM field. - event.idm.read_only_udm.metadata.event_timestamp: Newly mapped `timestamp` raw log field with `event.idm.read_only_udm.metadata.event_timestamp` UDM field. |
| 2025-06-27 | Enhancement:
- Added grok patterns to handle the dropped logs. - Renamed `SOAPAction` to `addition.SOAPAction`, `externalId` to `addition.externalId`, `attempts` to `addition.attempts`, `tty` to `addition.tty`, `audit_component` to `addition.audit_component`, `errdefs_msgno` to `addition.errdefs_msgno`, `route_domain` to `addition.route_domain`, `profile_name` to `addition.profile_name`, `action` to `addition.action`, `previous_action` to `addition.previous_action`, `bot_signature` to `addition.bot_signature`, `bot_signature_category` to `addition.bot_signature_category`, `bot_name` to `addition.bot_name`, `class` to `addition.class`, `anomaly_categories` to `addition.anomaly_categories`, `anomalies` to `addition.anomalies`, `micro_service_name` to `addition.micro_service_name`, `micro_service_type` to `addition.micro_service_type`, `micro_service_matched_wildcard_url` to `addition.micro_service_matched_wildcard_url`, `micro_service_hostname` to `addition.micro_service_hostname`, `browser_configured_verification_action` to `addition.browser_configured_verification_action`, `browser_actual_verification_action` to `addition.browser_actual_verification_action`, `new_request_status` to `addition.new_request_status`, `enforced_by` to `addition.enforced_by`, `mobile_is_app` to `addition.mobile_is_app`, `challenge_failure_reason` to `addition.challenge_failure_reason`, `client_type` to `addition.client_type`, `application_display_name` to `addition.application_display_name`, `Accept-Language` to `addition.Accept-Language`, `Content-Type` to `addition.Content-Type`, `support_id` to `addition.support_id`, `form_data` to `addition.form_data`, `query_string` to `addition.query_string`, `req_status` to `addition.req_status`, `resp` to `addition.resp`, `violate_rate` to `addition.violate_rate`, `ip_addr_intelli` to `addition.ip_addr_intelli`, `geo_info` to `addition.geo_info`, `websocket_message_type` to `addition.websocket_message_type`, `Cookie` to `addition.Cookie`, `Accept-Encoding` to `addition.Accept-Encoding`, `Accept-Charset` to `addition.Accept-Charset`, `Keep-Alive` to `addition.Keep-Alive`, `Connection` to `addition.Connection`, `Pragma` to `addition.Pragma`, `Cache-Control` to `addition.Cache-Control`, `Accept` to `addition.Accept`, `sub_violations` to `addition.sub_violations`, `and violation_rating` to `addition.violation_rating`. - Updated the mapping of `event.idm.read_only_udm.additional.fields` to utilize a generalized map for fields `SOAPAction`, `support_id`, `externalId`, `attempts`, `tty`, `audit_component`, `errdefs_msgno`, `route_domain`, `profile_name`, `action`, `previous_action`, `bot_signature`, `bot_signature_category`, `bot_name`, `class`, `anomaly_categories`, `anomalies`, `micro_service_name`, `micro_service_type`, `micro_service_matched_wildcard_url`, `micro_service_hostname`, `browser_configured_verification_action`, `browser_actual_verification_action`, `new_request_status`, `enforced_by`, `mobile_is_app`, `challenge_failure_reason`, `client_type`, `application_display_name`, `Accept-Language`, `Content-Type`, `form_data`, `query_string`. - Updated the mapping of `event.idm.read_only_udm.security_result.detection_fields` to utilize a generalized map for fields `req_status`, `resp`, `violate_rate`, `ip_addr_intelli`, `geo_info`, `websocket_message_type`, `Cookie`. - Updated the mapping of `event.idm.read_only_udm.security_result.about.resource.attribute.labels` to utilize a generalized map for fields `Accept-Encoding`, `Accept-Charset`, `Keep-Alive`, `Connection`, `Pragma`, `Cache-Control`, `Accept`, `sub_violations`, `violation_rating`. - Consolidate the mapping of `severity` and `level` to eliminate redundant code. - Removed redundant mapping of `event.idm.read_only_udm.security_result`. - Removed redundant mapping of `event.idm.read_only_udm.network.http.response_code` and used common field `network_http_response_code` and mapped it to `event.idm.read_only_udm.network.http.response_code`. - Removed redundant mapping of `event.idm.read_only_udm.target.port` and used common field `target_port` and mapped it to `event.idm.read_only_udm.target.port`. - Removed redundant mapping of `event.idm.read_only_udm.principal.port` and used common field `principal_port` and mapped it to `event.idm.read_only_udm.principal.port`. - Removed redundant code for field `attack_type`. - Renamed `security_result` to `sec_result`. - If `has_principal` is `true` and `has_target` is `true`, then set `event.idm.read_only_udm.metadata.event_type` to `NETWORK_HTTP` and `event.idm.read_only_udm.network.application_protocol` to `HTTP`. - If `has_target` is `false` and `has_principal` is `true`, then set `event.idm.read_only_udm.metadata.event_type` to `STATUS_UPDATE`. |
| 2025-06-02 | Enhancement:
- event.idm.read_only_udm.security_result.action_details, event.idm.read_only_udm.security_result.action: Newly mapped `req_status` raw log field with `event.idm.read_only_udm.security_result.action` and `event.idm.read_only_udm.security_result.action_details` UDM field. - if `act` is `alerted` and `cn1` is not `0` then set `event.idm.read_only_udm.security_result.action` to `ALLOW`. - event.idm.read_only_udm.additional.fields: Newly mapped `deviceCustomDate1` and `deviceCustomDate1Label` raw log field with `event.idm.read_only_udm.additional.fields` UDM field. - event.idm.read_only_udm.principal.ip, event.idm.read_only_udm.principal.asset.ip: Added grok pattern to extract IP address from `c6a2` raw log field then mapped `source_ip` with `event.idm.read_only_udm.principal.ip` and `event.idm.read_only_udm.principal.asset.ip` UDM field if `c6a2Label` is `source_address`. - event.idm.read_only_udm.metadata.event_type: If principal data and target data is present then set `event.idm.read_only_udm.metadata.event_type` to `NETWORK_CONNECTION`. - Added grok pattern for `cs3` raw log field to extract `incap_client_ip` and `refer_url` UDM field. - event.idm.read_only_udm.principal.asset.ip, event.idm.read_only_udm.principal.ip: Newly mapped `incap_client_ip` raw log field with `event.idm.read_only_udm.principal.ip` and `event.idm.read_only_udm.principal.asset.ip` UDM field. - event.idm.read_only_udm.network.http.referral_url: Newly mapped `refer_url` raw log field with `event.idm.read_only_udm.network.http.referral_url` UDM field. |
| 2025-05-21 | Enhancement:
- Added a new grok pattern in order to parse log with `kv_data_1` and `kv_data_2` raw log fields. - Added a new kv filter for `kvdata1` and `kvdata2` raw log fields. |
| 2025-04-16 | Enhancement:
- Added a Grok pattern to parse "event.idm.read_only_udm.network.http.user_agent" UDM field from "cs5" raw log field. |
| 2025-04-02 | Enhancement:
- Modified and added few gsubs to ensure proper parsing of KV format logs. - Added IP check for field "src" using grok before mapping it to "principal.nat_ip". |
| 2025-03-10 | Enhancement:
- Mapped "cn1" to "additional.fields". - Mapped "security_result.action" to "BLOCK" when "cn1" is "0". |
| 2025-02-11 | Enhancement:
- Mapped "column3" to "principal.ip" and "principal.asset.ip" |
| 2025-02-11 | Enhancement:
- Mapped "column3" to "principal.ip" and "principal.asset.ip" |
| 2025-02-04 | Enhancement:
- Added a "gsub" to remove non-utf8 characters from "uri" field when it contains non-utf8 characters to parse logs. |
| 2025-01-30 | Enhancement:
- Removed the "cs5" field from "_intermediary.ip" and "_intermediary.asset.ip". - Mapped "src" to "principal.nat_ip". - Mapped "cs5" to "principal.ip" and "principal.asset.ip". |
| 2025-01-17 | Enhancement:
- Removed the drop condition to parse logs with non-utf8 characters. |
| 2024-12-11 | Enhancement:
- Modified a Grok Pattern to support a new format of syslog logs. |
| 2024-11-28 | Enhancement:
- Changed the mapping of "Referer" from "network.http.referral_url" to "target.url". |
| 2024-11-07 | Enhancement:
- Mapped "exec_data" to "target.process.command_line". - Mapped "src" to "principal.hostname" and "principal.asset.hostname". - Mapped "cs3" to "additional.fields". |
| 2024-10-30 | Enhancement:
- Added support to handle CSV logs. |
| 2024-10-28 | Enhancement:
- Modified existing Grok pattern to handle ISP block and ISP GEO block. |
| 2024-10-25 | Enhancement:
- Mapped "form_data" to "additional.fields". |
| 2024-10-23 | Enhancement:
- Mapped "SOAPAction" to "additional.fields". |
| 2024-09-30 | Enhancement:
- Mapped "link" to "target.url" - When the message contains "DROP" then set "security_result.action" to "BLOCK". - When the message contains "allowed" then set "security_result.action" to "ALLOW". |
| 2024-08-07 | Enhancement:
- Modified existing Grok pattern to handle CEF logs. - Mapped "suid" to "principal.user.userid". - Mapped "suser" to "principal.user.user_display_name". - Mapped "device_version" to "metadata.product_version". - Mapped "severity" to "security_result.severity". |
| 2024-07-15 | Enhancement:
- Added support to handle the SYSLOG + KV logs. |
| 2024-06-17 | Enhancement:
- Added support for a new pattern of CSV logs. |
| 2024-06-11 | Enhancement:
- Added KV block to handle unparsed KV logs. - Formatted CSV logs using "gsub" to parse CSV logs. |
| 2024-05-13 | Enhancement:
- Added KV block to parse KV logs. - Added "gsub" to remove unwanted characters. |
| 2024-04-19 | Enhancement:
- Handled CSV unparsed logs. - Added a Grok pattern to map "resp_code". - Mapped "errdefs_msgno", "support_id_array", "audit_component" to "additional.fields". - Mapped "descrip" to "metadata.description". |
| 2024-04-08 | Enhancement:
- Added support to parse newly ingested unparsed logs. |
| 2024-04-05 | Bug-Fix:
- Added condition to parse dropped ASF CEM logs. |
| 2024-02-27 | Bug-Fix:
- When "cs5" field has a valid IP address, then mapped to "principal.ip". - Aligned "principal.ip" and "principal.asset.ip" mappings. - Aligned "principal.hostname" and "principal.asset.hostname" mappings. - Aligned "target.ip" and "target.asset.ip" mappings. - Aligned "target.hostname" and "target.asset.hostname" mappings. |
| 2024-01-12 | Enhancement:
- Mapped "severity" to "security_result.severity_details". - Mapped "resp_code" to "http.response_code". - Mapped "virus_name" to "security_result.threat_name". - Mapped "ip_route_domain" to "principal.ip". - Mapped "geo_info", "resp", "req_status", "violate_rate", and "ip_addr_intelli" to "security_result.detection_fields". |
| 2023-12-15 | Enhancement:
- Handled newly ingested set of logs where "metadata.event_type" is "GENERIC_EVENT" and "network.application_protocol" is "HTTP". - Set "network.ip_protocol" to "UDP" if message contains "UDP". - Removed hardcoding value of "network.application_protocol". - Set "network.application_protocol" to "HTTP" and "HTTPS" if "message" has "HTTP" and "HTTPS, respectively. - Set "network.application_protocol" to "HTTP" if "metadata.event_type" is "NETWORK_HTTP". - Added two Grok patterns to parse "principal_ip" and "src_port" from newly ingested logs. - Mapped "message_body" to "metadata.description". - Mapped "tmm_msg" to "metadata.description" |
| 2023-12-07 | Enhancement:
- Added a new Grok pattern to parse new KV+XML logs. - Added KV filters to parse unparsed KV logs. - Added XML filters to parse unparsed XML logs. - Mapped "policy_name" to "security_result.about.resource.name". - Mapped "viol_name" to "security_result.detection_fields". - Mapped "response_code" to "network.http.response_code". - Modified Grok pattern to map complete "Referer" field to "network.http.referral_url". - Mapped "parseduseragent" to "network.http.parsed_user_agent. |
| 2023-11-08 | Enhancement:
- Added a new Grok pattern to parse new KV logs. - Added a KV filter to parse uparsed KV logs. - Mapped "bigip_mgmt_ip", "client_ip_geo_location", "client_port", "client_request_uri", "device_version", "http_method", "route_domain" and "virtual_server_name" to "principal.ip", "principal.location.country_or_region", "principal.port", "principal.url", "metadata.product_version", "network.http.method", "additional.fields", "network.tls.client.server_name", respectively. - Added "legal" to "request_status" condition to map "security_result.action_details" as "ALLOW". - Mapped "profile_name", "action", "previous_action", "bot_signature", "bot_signature_category", "bot_name", "class", "anomaly_categories", "anomalies", "micro_services_name", "micro_services_type", "micro_services_matched_wildcard_url", "micro_services_hostname", "browser_configured_verification_action", "browser_actual_verification_action", "new_request_status", "mobile_is_app", "enforced_by", "application_display_name", "client_type", and "challenge_failure_reason" to "additional.fields". |
| 2023-10-19 | Enhancement:
- Added a Grok pattern to extract the value of "Referer" field as "referer" from CEF logs. - Mapped "referer" to "network.http.referral_url". |
| 2023-09-27 | Bug-Fix:
- Set "security_result.action" to "BLOCK" and "security_result.action_details" to "blocked" for logs having "request_status = blocked". - Set "security_result.action" to "ALLOW" and "security_result.action_details" to "passed" for logs having "request_status = passed". - Set "security_result.action" to "QUARANTINE" and "security_result.action_details" to "alerted" for logs having "request_status = alerted". |
| 2023-08-07 | Enhancement:
- Mapped "management_ip_address" to "metadata.intermediary.ip". - Mapped "request_status" to "security_result.action". - Mapped "query_string" to "additional.fields". - Mapped "sig_ids" to "security_result.rule_id". - Mapped "sig_names" to "security_result.rule_name". - Mapped "username" to "principal.user.userid". - Mapped "policy_name" to "security_result.about.resource.name". - Mapped "sub_violations" to "security_result.about.resource.attribute.labels". - Mapped "violation_rating" to "security_result.about.resource.attribute.labels". - Mapped "websocket_direction" to "network.direction". - Mapped "websocket_message_type" to "security_result.detection_fields". |
| 2023-07-27 | Bug-Fix:
- Added a new field "target_app" to contain value corresponding to "target.application". - Mapped the field "process" to "target.application" only when value of the field "target_app" is null. - Converted the field "process" to "string" if it's already not a string. |
| 2023-07-03 | Enhancement:
- Mapped "externalId" to ""additional.fields". - Mapped the event time to ""metadata.event_timestamp". |
| 2023-05-12 | Enhancement - For CEF format logs, mapped the information about the attack to "security_result.description".
|
| 2023-04-06 | Enhancement:
- Login event parsed as 'USER_LOGIN' instead of 'STATUS UPDATE'. - Parsed the username value in 'firstname.lastname' and mapped to 'principal.user.userid'. |
| 2023-02-09 | Enhancement- Parsed the logs containing "type=irule" by adding new grok pattern and mapped the following fields:
- Mapped "type" to "metadata.product_event_type". - Mapped "data.sessionid" to "network.session_id". - Mapped "data.bits" to "network.sent_bytes". - Mapped "data.version" to "network.tls.version". - Mapped "client_ip" to "principal.ip". - Mapped "client_port" to "principal.port". - Mapped "snat_ip" to "principal.nat_ip". - Mapped "snat_port" to "principal.nat_port". - Mapped "server_ip" to "target.ip". - Mapped "server_port" to "target.port". - Mapped "irule" to "security_result.rule_name". - Mapped "irule-version" to "security_result.rule_version". - Mapped "proxy_id" to "security_result.rule_id". - Mapped "virtualserver" to "network.tls.client.server_name". |
| 2022-11-03 | Enhancement:
- Added a condition for unparsed CEF format logs. - Added a condition to check for sshd and httpd user_login logs. - Added grok patterns to parse httpd and sshd user_login success/failure logs. - Mapped "event_id" to "metadata.product_log_id". - Mapped "application" to "target.application". - Mapped "prin_ip" to "principal.ip". - Mapped "SSH" to "app_protocol" when "tty" is "ssh" or "applicaition" is "sshd". - Mapped "user_id" "principal.user.user_id". - Mapped "USER_LOGIN" to "metadata.event_type" for httpd/sshd user_login logs. - Mapped "auth_level" to "principal.user.attribute.roles". - Mapped "addr" from log to "target.ip" - Mapped "port" from log to "target.port" |
| 2022-09-21 | Enhancement:
- Migrated customer specific to default parser. |
| 2022-05-17 | Enhancement: Enhanced the parser to parse the header of the HTTP request.
|
| 2022-04-27 | Bug - Fix:
- Enhanced the parser to parse logs with the "ASM:" format. |
| 2022-04-26 | Enhanced the parser to handle unparsed raw logs |