Collect Microsoft IIS logs

Supported in:

This document explains how to collect Microsoft Internet Information Services (IIS) logs to Google Security Operations by using Bindplane. The parser first attempts to cleanse and normalize the input data by removing unnecessary characters and standardizing field names. Then, it uses a series of grok patterns to extract relevant fields from various Microsoft IIS log formats and maps them to the unified data model (UDM).

Before you begin

  • Ensure that you have a Google SecOps instance.
  • Ensure that you have a Windows 2016 or later.
  • If running behind a proxy, ensure firewall ports are open.

Get Google SecOps ingestion authentication file

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Collection Agents.
  3. Download the Ingestion Authentication File. Save the file securely on the system where Bindplane will be installed.

Get Google SecOps customer ID

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Profile.
  3. Copy and save the Customer ID from the Organization Details section.

Install the Bindplane agent on Windows

  1. Open the Command Prompt or PowerShell as an administrator.

  2. Run the following command:

    msiexec /i "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" /quiet

Additional installation resources

Configure the Bindplane agent to ingest Syslog and send to Google SecOps

  1. Before configuring the YAML file, stop the observIQ Distro for Open Telemetry Collector Service in the Services Panel.

  2. Access the configuration file:

    1. Locate the config.yaml file. Typically, it's in the /etc/bindplane-agent/ directory on Linux or in the installation directory on Windows.
    2. Open the file using a text editor (for example, nano, vi, or Notepad).

  3. Edit the config.yaml file as follows:

    receivers:
  iis:
    collection_interval: 60s

processors:
  # Resourcedetection is used to add a unique (host.name) to the metric resource(s), allowing users to filter between multiple agent systems.
  resourcedetection:
    detectors: ["system"]
    system:
      hostname_sources: ["os"]

  normalizesums:

  batch:

exporters:
  chronicle/powershell:
    endpoint: malachiteingestion-pa.googleapis.com
    # Adjust the path to the credentials file you downloaded in Step 1
    creds: '/path/to/ingestion-authentication-file.json'
    log_type: 'IIS'
    override_log_type: false
    raw_log_field: body
    customer_id: '<customer_id>'

service:
  pipelines:
    logs/winpowershell:
      receivers:
        - iis
      processors:
        - resourcedetection
        - normalizesums
        - batch
      exporters: [chronicle/iis]

  4. Replace <customer_id> with the actual customer ID.

  5. Update /path/to/ingestion-authentication-file.json to the path where the authentication file was saved in the Get Google SecOps ingestion authentication file section.

  6. After saving the config.yaml file, start the observIQ Distro for Open Telemetry Collector Service.

Restart the Bindplane agent to apply the changes

  • To restart the Bindplane agent in Windows, you can either use the Services console or enter the following command:

    net stop BindPlaneAgent && net start BindPlaneAgent

UDM Mapping Table

Log field UDM mapping Logic
@timestamp metadata.event_timestamp The timestamp of the event as recorded in the raw log.
@version metadata.product_version The version of the IIS server.
AgentDevice additional.fields.AgentDevice.value.string_value The device that generated the log.
AgentLogFile additional.fields.AgentLogFile.value.string_value The name of the log file.
ASP.NET_SessionId network.session_id The session ID of the user.
c-ip principal.ip The IP address of the client.
Channel security_result.about.resource.attribute.labels.Channel.value The channel where the event was logged.
ChannelID security_result.about.resource.attribute.labels.ChannelID.value The ID of the channel where the event was logged.
Computer target.hostname The hostname of the target machine.
cs-bytes network.received_bytes The number of bytes received from the client.
cs-host principal.hostname, principal.asset.hostname The hostname of the client.
cs-method network.http.method The HTTP method used by the client.
cs-uri-query target.url The query string of the URL requested by the client.
cs-uri-stem target.url The path of the URL requested by the client.
cs-username principal.user.user_display_name The username of the client.
cs-version network.tls.version_protocol The HTTP version used by the client.
cs(Cookie) Used to extract cookie information.
cs(Referer) network.http.referral_url The URL that referred the client to the current page.
cs(User-Agent) network.http.user_agent The user agent of the client.
csbyte network.received_bytes The number of bytes received from the client.
cshost principal.hostname, principal.asset.hostname The hostname of the client.
csip principal.ip, principal.asset.ip The IP address of the client.
csmethod network.http.method The HTTP method used by the client.
csreferer network.http.referral_url The URL that referred the client to the current page.
csuseragent network.http.user_agent The user agent of the client.
csusername principal.user.user_display_name The username of the client.
csversion network.tls.version_protocol The HTTP version used by the client.
date Used to construct the event timestamp if the raw log timestamp is invalid.
description security_result.description A description of the event.
devicename target.hostname The hostname of the target machine.
dst_ip target.ip, target.asset.ip The IP address of the target machine.
dst_port target.port The port number of the target machine.
duration The duration of the request in milliseconds.
EventEnqueuedUtcTime additional.fields.EventEnqueuedUtcTime.value.string_value The time when the event was enqueued in UTC.
EventID metadata.product_log_id The ID of the event.
EventProcessedUtcTime additional.fields.EventProcessedUtcTime.value.string_value The time when the event was processed in UTC.
EventTime metadata.event_timestamp The timestamp of the event.
EventType metadata.product_event_type The type of the event.
file_path target.file.full_path The full path of the file involved in the event.
FilterId security_result.about.resource.attribute.labels.FilterId.value The ID of the filter.
FilterKey security_result.about.resource.attribute.labels.FilterKey.value The key of the filter.
FilterName security_result.about.resource.attribute.labels.FilterName.value The name of the filter.
FilterType security_result.about.resource.attribute.labels.FilterType.value The type of the filter.
host target.hostname The hostname of the target machine.
host.architecture principal.asset.hardware.cpu_platform The architecture of the host machine.
host.geo.name additional.fields.geo_name.value.string_value The geographical location of the host machine.
host.hostname target.hostname, target.asset.hostname The hostname of the host machine.
host.id observer.asset_id The ID of the host machine.
host.ip principal.ip, principal.asset.ip The IP address of the host machine.
host.mac principal.mac The MAC address of the host machine.
host.os.build additional.fields.os_build.value.string_value The build number of the operating system on the host machine.
host.os.kernel principal.platform_patch_level The kernel version of the operating system on the host machine.
host.os.name additional.fields.os_name.value.string_value The name of the operating system on the host machine.
host.os.platform principal.platform The platform of the operating system on the host machine.
host.os.version principal.platform_version The version of the operating system on the host machine.
http_method network.http.method The HTTP method used by the client.
http_response network.http.response_code The HTTP response code.
http_status_code network.http.response_code The HTTP status code of the response.
http_substatus additional.fields.sc_substatus.value.string_value The HTTP substatus code of the response.
instance additional.fields.instance.value.string_value The instance ID of the task.
intermediary_devicename intermediary.hostname, intermediary.asset.hostname The hostname of the intermediary device.
json_message The raw log message in JSON format.
kv_fields Used to extract key-value pairs from the raw log message.
LayerKey security_result.about.resource.attribute.labels.LayerKey.value The key of the layer.
LayerName security_result.about.resource.attribute.labels.LayerName.value The name of the layer.
LayerId security_result.about.resource.attribute.labels.LayerId.value The ID of the layer.
log.file.path target.file.full_path The full path of the log file.
log.offset metadata.product_log_id The offset of the event in the log file.
logstash.collect.host observer.hostname The hostname of the machine that collected the log.
logstash.process.host intermediary.hostname The hostname of the machine that processed the log.
logstash_json_message The raw log message in JSON format.
message security_result.description The raw log message.
ministry additional.fields.ministry.value.string_value The ministry associated with the event.
name The name of the entity.
NewValue additional.fields.NewValue.value.string_value The new value of the configuration setting.
OldValue additional.fields.OldValue.value.string_value The old value of the configuration setting.
port principal.port The port number of the client.
priority_code The priority code of the syslog message.
ProcessID principal.process.pid The process ID of the process that generated the event.
ProviderGuid security_result.about.resource.attribute.labels.ProviderGuid.value The GUID of the provider.
ProviderKey security_result.about.resource.attribute.labels.ProviderKey.value The key of the provider.
ProviderName security_result.about.resource.attribute.labels.ProviderName.value The name of the provider.
referrer_url network.http.referral_url The URL that referred the client to the current page.
request_url target.url The URL requested by the client.
s-computername target.hostname The hostname of the target machine.
s-ip target.ip, target.asset.ip The IP address of the target machine.
s-port target.port The port number of the target machine.
s-sitename additional.fields.sitename.value.string_value The name of the site.
sc-bytes network.sent_bytes The number of bytes sent to the client.
sc-status network.http.response_code The HTTP status code of the response.
sc-substatus additional.fields.sc_substatus.value.string_value The HTTP substatus code of the response.
sc-win32-status The Windows status code of the response.
scbyte network.sent_bytes The number of bytes sent to the client.
scstatus network.http.response_code The HTTP status code of the response.
severity security_result.severity The severity of the event.
service.type additional.fields.service_type.value.string_value The type of the service.
sIP principal.ip, principal.asset.ip The IP address of the client.
sPort principal.port The port number of the client.
sSiteName additional.fields.sitename.value.string_value The name of the site.
src_ip principal.ip, principal.asset.ip, observer.ip The IP address of the client.
src_port principal.port The port number of the client.
sysdate The date and time of the syslog message.
syslog_facility security_result.severity_details The facility of the syslog message.
syslog_pri The priority of the syslog message.
syslog_severity security_result.severity_details The severity of the syslog message.
syslog_severity_code The severity code of the syslog message.
tags security_result.rule_name Tags associated with the event.
task additional.fields.task.value.string_value The name of the task.
time Used to construct the event timestamp if the raw log timestamp is invalid.
time-taken The duration of the request in milliseconds.
uri_query target.url The query string of the URL requested by the client.
user_agent network.http.user_agent The user agent of the client.
UserName target.user.userid The username of the user.
UserSid target.user.windows_sid The Windows SID of the user.
Weight security_result.about.resource.attribute.labels.Weight.value The weight of the filter.
win32_status The Windows status code of the response.
xforwardedfor The X-Forwarded-For header, containing a comma-separated list of IP addresses.
metadata.log_type IIS
network.direction INBOUND
metadata.vendor_name Microsoft
metadata.product_name Internet Information Server
metadata.event_type NETWORK_HTTP, USER_UNCATEGORIZED, GENERIC_EVENT, STATUS_UPDATE, USER_LOGOUT, USER_LOGIN
extensions.auth.type MACHINE

Changes

2025-01-27

Enhancement:

  • Added support to parse XML logs.

2024-06-20

Enhancement:

  • Mapped streamid to additional.fields.

2024-06-11

Enhancement:

  • Added a Grok pattern to parse unparsed logs.

2024-02-18

Enhancement:

  • Modified Grok pattern field names to map the value correctly.
  • Modified name from intermediary_devicename to observer_devicename.
  • Modified name from src_ip to observer_ip.
  • Modified name from dst_ip to principal_or_intermediary_ip.
  • Modified name from xff_ips to principal_xff_ips.
  • Mapped observer_ip to observer.ip.
  • Mapped observer_devicename to observer.hostname.
  • Mapped principal_xff_ips to principal.ip.
  • If principal_xff_ips is present, then mapped principal_or_intermediary_ip to intermediary.ip.
  • If principal_xff_ips is not present, then mapped principal_or_intermediary_ip to principal.ip.

2024-02-12

Enhancement:

  • Added new Grok patterns to support the network logs with kv data.
  • Mapped sc-substatus to additional.fields.

2024-01-09

Enhancement:

  • Added new Grok patterns to support the network logs with key-value data.
  • Mapped dst_ip2 to target.ip.
  • Mapped X-BackEndCookie,exchangecookie, OutlookSession, MapiContext, MapiRouting, content_type and MapiSequence to additional.fields.

2023-12-19

Enhancement:

  • Mapped Configuration as a value to metadata.product_event_type if EventType is equal to VERBOSE.
  • Mapped Message to security_result.description.
  • Mapped PhysicalPath to target.file.full_path.
  • Mapped OldValue, NewValue, Configuration and ConfigPath to additional.fields.

2023-12-06

Enhancement:

  • Mapped @timestamp to metadata.event_timestamp.
  • Mapped host.hostname to target.hostname.
  • Mapped logstash.process.host to intermediary.hostname.
  • Mapped logstash.collect.host to observer.hostname.
  • Mapped _user to principal.user.userid.
  • Mapped http_response to network.http.response_code.
  • Mapped referer to network.http.referral_url.
  • Mapped syslog_severity to security_result.severity_details.
  • Mapped message to security_result.description.
  • When request_url is /login.aspx and entity is AutoLogout=1, then set metadata.event_type to USER_LOGOUT.
  • When request_url is /login.aspx, then set metadata.event_type to USER_LOGIN.

2023-10-27

Enhancement:

  • Mapped Noun.hostname and Noun.asset.hostname to the same value.
  • Mapped cIP to target.ip.
  • Mapped csUriStem to `target.url.
  • Mapped sPort to principal.port.
  • Mapped csUserAgent to http.user_agent.
  • Mapped sIP to principal.ip.
  • Mapped csMethod to network.http.method.
  • Mapped scStatus to http.response_code.
  • Mapped sComputerName to target.hostname.
  • Mapped _resourceId to target.resource.id.
  • Mapped scBytes to network.sent_bytes.
  • Mapped csBytes to network.received_bytes.
  • Mapped sSiteName, TenantId, EventProcessedUtcTime, ManagementGroupName, and EventEnqueuedUtcTime to additional.fields.
  • Mapped TimeGenerated to about.resource.attribute.labels.
  • Mapped SourceSystem to security_result.detection_fields.

2023-06-23

Bug fix:

  • Change in log Format
  • Defined the grok for the changed log pattern
  • Mapped cs-host to principal.application
  • Mapped uristem & cs-uri-stem using if-else to target.url
  • Mapped cs-version to network.tls.version_protocol
  • Replaced the value of severity with the value in the field Level
  • Replaced the value of sitename with s-sitename
  • Replaced the value of UserSid with UserID
  • Mapped AgentDevice as label key and value
  • Mapped app_name to principal.application
  • Mapped ChannelID to security_result.about.resource.attribute
  • Mapped Level to security_result.severity
  • Mapped ExecutionProcessID to principal.process.pid
  • Mapped ExecutionThreadID to principal.process.product_specific_process_id
  • Mapped Domain to principal.user.userid
  • Mapped UserID to principal.user.windows_sid
  • Mapped AccountType to principal.user.role_name

2023-05-12

Enhancement:

  • Parsed failing JSON logs
  • Mapped cshost to principal.hostname
  • Mapped csusername to principal.user.user_display_name.
  • Mapped sip to target.ip.
  • Mapped uristem to target.url.
  • Mapped sport to target.port.
  • Mapped csversion to network.tls.version_protocol.
  • Mapped csuseragent to network.http.user_agent.
  • Mapped csip to principal.ip.
  • Mapped xforwardedfor to principal.ip.
  • Mapped csmethod to network.http.method.
  • Mapped csreferer to network.http.referral_url.
  • Mapped scstatus to network.http.response_code.
  • Mapped computername to target.asset.hostname.
  • Mapped scbyte to network.sent_bytes.
  • Mapped csbyte to network.received_bytes.
  • Mapped date and time to metadata.event_timestamp.

2023-03-01

Enhancement:

  • Added support for JSON format logs having keys like: c-*, s-*, cs-*, sc-*.

2022-10-25

Enhancement:

  • Extracted uri_query,intermediary_device_name,principal_username,sent_bytes,received_bytes from log.
  • Changed dst_port to src_port.
  • Mapped src_port to principal.port.
  • Updated target.url mapping to request_url+?+uri_query.
  • Mapped token_data to security_result.detection_fields as Cookie.
  • Mapped sent_bytes to network.sent_bytes.
  • Mapped received_bytes to network.received_bytes.
  • Mapped intermediary_devicename to target.hostname if target_host is empty, else mapped it to intermediary.hostname.
  • Mapped principal_username to principal.user.userid.
  • Mapped Devicename to target.hostname only when target_host is empty.
  • Mapped src_ip to principal.ip if xff_ips is empty.
  • If xff_ips is not empty:
  • Mapped src_ip to intermediary.ip.
  • Mapped xff_ips to principal.ip.
  • Added a Grok pattern for additional logs types.

2022-10-01

Enhancement:

  • Wrote grok to parse unparse syslogs`.
  • Mapped Severity to security_result.severity.
  • Mapped UserName to target.user.userid.
  • Mapped UserSid to target.user.windows_sid.
  • Mapped ProviderKey to security_result.about.resource.attribute.labels.
  • Mapped LayerKey to security_result.about.resource.attribute.labels.
  • Mapped LayerName to security_result.about.resource.attribute.labels.
  • Mapped LayerId to security_result.about.resource.attribute.labels.
  • Mapped Weight to security_result.about.resource.attribute.labels.
  • Mapped Conditions to security_result.about.resource.attribute.labels.
  • Mapped Action to security_result.about.resource.attribute.labels.
  • Mapped CalloutKey to security_result.about.resource.attribute.labels.
  • Mapped CalloutName to security_result.about.resource.attribute.labels.
  • Mapped Channel to security_result.about.resource.attribute.labels.
  • Mapped FilterId to security_result.about.resource.attribute.labels.
  • Mapped FilterKey to security_result.about.resource.attribute.labels.
  • Mapped FilterName to security_result.about.resource.attribute.labels.
  • Mapped FilterType to security_result.about.resource.attribute.labels.
  • Mapped ProviderGuid to security_result.about.resource.attribute.labels.
  • Mapped ProviderName to security_result.about.resource.attribute.labels.
  • Mapped SourceName to security_result.about.resource.attribute.labels.
  • Mapped SyslogSeverity to security_result.about.resource.attribute.labels.
  • Mapped Category to security_result.category_details.
  • Mapped EventType to metadata.product_event_type.
  • Mapped EventID to metadata.product_log_id.
  • Mapped ProcessID to principal.process.pid.
  • Mapped SourceModuleName to target.resource.name.
  • Mapped SourceModuleType to observer.application.

2022-09-30

Enhancement:

  • Mapped xff header IPs to intermediary.ip.
  • Mapped hostname to target.hostname.
  • Added extra grok pattern to correctly parse certain logs with extra information and Mapped that extra information in additional fields.
  • Mapped ASP.NET_Session_id to network.session_id.

2022-03-30

Enhancement:

  • Parse additional fields.

Need more help? Get answers from Community members and Google SecOps professionals.