Change log for BLUECOAT_WEBPROXY
| Date | Changes |
|---|---|
| 2025-06-24 | Enhancement:
- Added new grok patterns to support the new format of SYSLOG logs. - `event.idm.read_only_udm.additional.fields` : Newly mapped 'transaction_id', and 'proxy_host' raw log fields with 'event.idm.read_only_udm.additional.fields' UDM field. - Modified the condition for mapping `event.idm.read_only_udm.metadata.event_type` to `NETWORK_HTTP` when `principal_present` and `target_present` are true. |
| 2025-06-18 | Enhancement:
- Added Grok patterns for new format of SYSLOG logs. - Added gsub to remove `\\\\\\"` from `cs_categories` and `x_bluecoat_application_name` raw log fields. - event.idm.read_only_udm.network.http.method: Newly mapped `cs_method` raw log field with `event.idm.read_only_udm.network.http.method` UDM field. - event.idm.read_only_udm.network.http.user_agent: Newly mapped `cs_user_agent` raw log field with `event.idm.read_only_udm.network.http.user_agent` UDM field. - event.idm.read_only_udm.network.http.referral_url: Newly mapped `cs_refer` raw log field with `event.idm.read_only_udm.network.http.referral_url` UDM field. - event.idm.read_only_udm.additional.fields: Newly mapped `rs_service_latency`, `x_icap_reqmod_header`, `x_icap_respmod_header`, `x_bluecoat_app_operation`, `x_bluecoat_total_time_added`, `policy_evaluation_time`, `x_bluecoat_appliance_name`, `cs_connection_ssl_server_name`, `rs_connection_ssl_server_name` raw log fields with `event.idm.read_only_udm.additional.fields` UDM field. - event.idm.read_only_udm.security_result.detection_fields: Newly mapped `x_data_leak_detected` raw log field with `event.idm.read_only_udm.security_result.detection_fields` UDM field. - event.idm.read_only_udm.target.ip , event.idm.read_only_udm.target.asset.ip: Newly mapped `r_Ip` raw log field with `event.idm.read_only_udm.target.ip` and `event.idm.read_only_udm.target.asset.ip` UDM field. - Added conditional check for `cs_uri_path` raw log field before mapping it to `event.idm.read_only_udm.target.file.full_path` UDM field. - Added conditional check for `sc_status` raw log field before mapping it to `event.idm.read_only_udm.network.http.response_code` UDM field. |
| 2025-06-09 | Enhancement:
- Added gsub for `message` to replace unwanted data. - Added grok pattern's to support new pattern of SYSLOG logs. - Modified mutate block from `rename` to `replace` for fields "csv.column4", "csv.column6", "csv.column9", "csv.column11", "csv.column12", "csv.column14", "csv.column15", "csv.column19", "csv.column20", "csv.column22", "csv.column27", "csv.column30", "csv.column31", "csv.column32", "csv.column34", "csv.column36", "csv.column37", and "csv.column39". - Added for loop for field `additional` and mapped it to `event.idm.read_only_udm.additional.fields`. - Removed redundant code for `vendor_action`, `s_action`, `s-action`, `cs_username`, `cs_auth_groups`, `cs_auth_group`, `cs-auth-group`, `cs_host`, `cs-host`, `cs_uri_path`, `r_ip`, `r-ip`, `dst`, `dst_ip`, `x_ip`, `dest_ip`, `rs_status`, `sc_status`, `sc-status`, `cs-bytes`, `cs_bytes` `sc-bytes`, `sc_bytes`, `useragent`, `http_user_agent`, `cs_user_agent`, `cs-user-agent`, `cs-method`, `cs_method`, `cs_uri_scheme`, `cs_categories`, and `principal_hostname`. - Added json_block to parse the json_data. |
| 2025-05-20 | Enhancement:
- Added a `Grok` filter to extract `date` and `time` from the `msg_attrs` field. - Implemented a `mutate` filter to create the `d_time` field by combining the extracted `date` and `time` values - event.idm.read_only_udm.metadata.event_timestamp: Mapped `d_time` field to `event.idm.read_only_udm.metadata.event_timestamp` UDM field. - event.idm.read_only_udm.principal.ip: Removed mapping of `s-ip` from `event.idm.read_only_udm.principal.ip` and `event.idm.read_only_udm.principal.asset.ip` UDM field. - event.idm.read_only_udm.intermediary.ip: Mapped `s-ip` raw log field with `event.idm.read_only_udm.intermediary.ip` UDM field. - event.idm.read_only_udm.additional.fields: Removed mapping of `cs-uri-port` from `event.idm.read_only_udm.additional.fields` UDM field. - event.idm.read_only_udm.target.port: Mapped `cs-uri-port` raw log field with `event.idm.read_only_udm.target.port` UDM field. - event.idm.read_only_udm.principal.hostname: Removed mapping of `cs-host` from `event.idm.read_only_udm.principal.hostname` and `event.idm.read_only_udm.principal.asset.hostname` UDM field. - event.idm.read_only_udm.target.hostname: Mapped `cs-host` raw log field with `event.idm.read_only_udm.target.hostname` and `event.idm.read_only_udm.target.asset.hostname` UDM field. - event.idm.read_only_udm.target.ip: Newly mapped `r-ip` raw log field with `event.idm.read_only_udm.target.ip` and `event.idm.read_only_udm.target.asset.ip` UDM field - event.idm.read_only_udm.target.ip: Removed mapping of `c-ip` from `event.idm.read_only_udm.target.ip` and `event.idm.read_only_udm.target.asset.ip` UDM field. - event.idm.read_only_udm.principal.ip: Mapped `c-ip` raw log field with `event.idm.read_only_udm.principal.ip` and `event.idm.read_only_udm.principal.asset.ip` UDM field. |
| 2025-05-14 | Enhancement:
- Added a GROK pattern to retrieve data `cs_username` and `cs_hostname`. |
| 2025-04-03 | Enhancement:
- Added a few gsubs to parse new format of logs. |
| 2025-03-20 | Enhancement:
- Added a GROK pattern to parse SYSLOG log. - Added Gsub for "ip" to remove brackets "\\]|\\[". - Added Gsub for "json_categories" to remove "\"\"". |
| 2025-03-13 | Enhancement:
- Mapped "cs-uri-path to "target.file.names". |
| 2025-02-19 | Enhancement:
- Added conditional check for "url". |
| 2025-01-10 | Enhancement:
- Mapped "dest_host" and "target_hostname" to "target.url". |
| 2025-01-09 | Enhancement:
- Added support for processing JSON logs in the new format. |
| 2024-12-23 | Enhancement:
- Added a new Grok pattern to validate the hostname. |
| 2024-12-10 | Enhancement:
- Removed mapping of "s-ip" from "target.ip" and "target.asset.ip". - Changed mapping of "x-sr-vpop-ip" from "principal.ip" and "principal.asset.ip" to "intermediary.ip". |
| 2024-12-05 | Enhancement:
- Mapped "target_url" to "target.url" only when its value is not "none". |
| 2024-11-14 | Enhancement:
- Mapped "proxy_name" and "column3" to "principal.asset.hostname". |
| 2024-10-25 | Enhancement:
- Added a new grok pattern to parse "cs_threat_risk" and "cs_categories". |
| 2024-10-18 | Enhancement:
- Added support to handle KV, CSV, and SYSLOG logs. |
| 2024-10-15 | Enhancement:
- Mapped "upload-source" to "additional.fields". |
| 2024-09-25 | Enhancement:
- Added support for new format logs. |
| 2024-09-11 | Enhancement:
- Set "metadata.event_type" to "NETWORK_HTTP" if "message" contains "SG - HTTP". |
| 2024-08-29 | Enhancement:
- Added support for a new log pattern. |
| 2024-08-22 | Enhancement:
- Added support for a new log pattern. - Added a Grok pattern to parse the new format of field "file_name". |
| 2024-08-07 | Enhancement:
- Mapped "time-taken" to "session_duration.session_duration". |
| 2024-06-20 | Enhancement:
- Added the new Grok patterns to parse new format of field "file_name". |
| 2024-06-18 | Enhancement:
- Added support to handle unparsed SYSLOG logs. |
| 2024-06-14 | Enhancement:
- Added support to parse dropped logs. |
| 2024-05-21 | Enhancement:
- Added a Grok pattern over "x_icap_respmod_header" to extract the fields "file_reputation" and "expect_sandbox". - Mapped "x_icap_respmod_header" to "security_result.detection_fields". - Mapped "file_reputation" to "security_result.detection_fields". - Mapped "expect_sandbox" to "security_result.detection_fields". |
| 2024-05-14 | Bug-Fix:
- Separated "principal_user_group_identifiers" CSV values and mapped them into "principal.user.group_identifiers". |
| 2024-05-09 | Enhancement:
- Parsed "search_query" from "target_url" and mapped it to "target.resource.attribute.labels". |
| 2024-05-06 | Bug-Fix:
- Mapped "cs_auth_groups" to "principal.user.group_identifiers". |
| 2024-04-25 | Bug-Fix:
- Removed "column16" mapping to "target.ip" as it is being mapped to "intermediary.ip". |
| 2024-02-21 | Enhancement:
- Added a Grok pattern to parse new format logs. |
| 2024-02-16 | Enhancement:
- Parsed "file_name" from "target.file.file_path" and mapped to "target.file.names". |
| 2024-02-06 | Enhancement:
- If "time_taken" is less than 1000, then mapped "time_taken" to "network.session_duration.nanos", else mapped to "network.session_duration.seconds". |
| 2024-01-25 | Enhancement:
- Mapped "x-tenant-id" to "security_result.detection_fields". |
| 2023-12-19 | Enhancement:
- Added mapping of "originating_ip" to "principal.ip". |
| 2023-12-13 | Bug-Fix:
- Changed mapping of "cs-host" from "principal.hostname" to "target.hostname". - Added null check to "c_ip_host" prior mapping to "principal.hostname". - Mapped "s-supplier-ip" to "intermediary.ip". - Mapped "s-source-ip" to "intermediary.ip". - Mapped "cs-uri-port" to "target.port". - Mapped "x-bluecoat-application-name" to "target.application". - Mapped "x-rs-certificate-validate-status" to "network.tls.server.certificate.subject". - Mapped "x-sr-vpop-country-code" to "principal.location.country_or_region". - Mapped "cs-icap-status" to "security_result.description". - Mapped "x-rs-ocsp-error", "x-cs-ocsp-error", "cs-icap-error-details", "rs-icap-error-details", "risk-groups", "x-rs-certificate-hostname-threat-risk", "cs-X-Requested-With", "x-rs-connection-negotiated-ssl-version", "x-cs-connection-negotiated-cipher-size", "x-rs-connection-negotiated-cipher-size", "x-rs-connection-negotiated-cipher", "x-bluecoat-reference-id", "x-bluecoat-placeholder", "wf_id", "verdict", "x-cloud-rs", "x-symc-dei-via", "x-sc-connection-issuer-keyring", "x-client-security-posture-risk-score", "s-supplier-failures", "x-data-leak-detected", "x-virus-id", "x-rs-certificate-observed-errors", "x-rs-connection-negotiated-cipher-strength", to "security_result.detection_fields". - When principal and target details are present, then set "metadata.event_type" to "NETWORK_CONNECTION". |
| 2023-11-27 | Enhancement:
- Added support for JSON logs. - Added on_error for mapping of "_network.http.response_code" to "network.http.response_code". - Initialized "date_time", "rs_status", "c_ip_host", "r_port", "json_message", and "r_dns" to null. - Added null check before mapping "rs_status" to "network.http.response_code". - Added null check to "date_time" before matching the date pattern. - Mapped "x-sr-vpop-ip" to "principal.ip". - Mapped "cs-userdn" to "principal.user.userid". - Mapped "x-client-agent-type" to "principal.application". - Mapped "x-client-agent-sw" to "principal.asset.software". - Mapped "x-sr-vpop-country" to "principal.location.country_or_region". - Mapped "x-client-device-id" to "principal.resource.product_object_id". - Mapped "application-name" to "target.applcation". - Mapped "rs_content_type" to "target.file.mime_type". - Mapped "sc_status" to "network.http.response_code". - Mapped "x-bluecoat-appliance-name" to "intermediary.application". - Mapped "s-supplier-country" to "intermediary.location.country_or_region". |
| 2023-11-13 | Enhancement-
- Mapped "rs_server" to "security_result.about.labels". - Mapped "c_uri_path_query" to "target.file.full_path". - Mapped "time_taken" to "network.session_duration.nanos". - Added "target_hostname" to complete "target_url", - Mapped "cs_threat_risk" to "security_result.risk_score". |
| 2023-10-01 | Enhancement-
- Removed dropping of logs that contain "Log uploading failed". - Added check to "ip_target" prior mapping "metadata.event_type" to "NETWORK_CONNECTION". If "ip_target" is "-" mapped "metadata.event_type" to "STATUS_UPDATE". - Logs parsed using CSV extraction instead of a Grok pattern. |
| 2023-08-18 | Enhancement-
- Added additional Grok pattern to parse the new format syslog logs. - Mapped 'x_cs_connection_negotiated_cipher' to 'network.tls.cipher'. - Mapped 'x_rs_certificate_hostname' to 'network.tls.client.server_name'. - Mapped 'x_rs_certificate_validate_status' to 'network.tls.server.certificate.subject'. - Mapped 's_icap_status' to 'security_result.description'. - Mapped 'x_cs_connection_negotiated_ssl_version' to 'network.tls.version'. |
| 2023-06-25 | Enhancement- Added a Grok pattern to parse unparsed logs.
- Changed "metadata.event_type" from 'GENERIC_EVENT' to a more specific value wherever possible. |
| 2023-04-27 | - Mapped "cs(User-Agent)" to "network.http.user_agent".
- Mapped "cs-uri-scheme" to "network.ip_protocol". - Added null checks to 'on_error' statements for some fields. - Mapped "dst_user" to "target.user.userid". - Mapped "session_id" to "network.session_id". - Added new Grok pattern for authentication log types. |
| 2022-09-28 | Enhancement - Migrated customer-specific parser to default.
- Added "on_error" statements while replacing the values of fields as they might not be present in the log. - Updated "metadata.event_type" to "NETWORK_CONNECTION" from "GENERIC_EVENT" wherever possible. - Added condition check before mapping "metadata.event_type" as "STATUS_UPDATE" or "STATUS_UNCATEGORIZED" to ensure value of "target.ip" or "target.hostname" is not present as otherwise it may throw an error. |
| 2022-08-23 | Enhancement -
- Mapped "sc_status" to "network.http.response_code". - Mapped "rule_name" to "security_result.rule_name". - Mapped "cs_method" to "network.http.method". - Mapped "application_protocol" to "network.application_protocol". - Mapped "communication_type" to "security_result.rule_name". - Mapped "rule_name" to "security_result.about.labels". - Added null check for "cs_host", "hostname", "cs_method", "cs_uri_scheme", "cs_username", "sc_bytes", "username". - Removed Drop statement. |
| 2022-05-25 | Enhancement - Added GROK extraction for PingSSOWAF syslog.
|
| 2022-04-20 | Enhancement - Dropped logs with improper JSON Format.
-on_error conditional checks are added to handle such logs. |