Collect Microsoft IIS logs
This document explains how to collect Microsoft Internet Information Services (IIS) logs to Google Security Operations by using Bindplane. The parser first attempts to cleanse and normalize the input data by removing unnecessary characters and standardizing field names. Then, it uses a series of grok
patterns to extract relevant fields from various Microsoft IIS log formats and maps them to the unified data model (UDM).
Before you begin
- Ensure that you have a Google SecOps instance.
- Ensure that you have a Windows 2016 or later.
- If running behind a proxy, ensure firewall ports are open.
Get Google SecOps ingestion authentication file
- Sign in to the Google SecOps console.
- Go to SIEM Settings > Collection Agents.
- Download the Ingestion Authentication File. Save the file securely on the system where Bindplane will be installed.
Get Google SecOps customer ID
- Sign in to the Google SecOps console.
- Go to SIEM Settings > Profile.
- Copy and save the Customer ID from the Organization Details section.
Install the Bindplane agent on Windows
- Open the Command Prompt or PowerShell as an administrator.
Run the following command:
msiexec /i "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" /quiet
Additional installation resources
- For additional installation options, consult this installation guide.
Configure the Bindplane agent to ingest Syslog and send to Google SecOps
- Before configuring the YAML file, stop the
observIQ Distro for Open Telemetry Collector
Service in the Services Panel. Access the configuration file:
- Locate the
config.yaml
file. Typically, it's in the/etc/bindplane-agent/
directory on Linux or in the installation directory on Windows. - Open the file using a text editor (for example,
nano
,vi
, or Notepad).
- Locate the
Edit the
config.yaml
file as follows:receivers: iis: collection_interval: 60s processors: # Resourcedetection is used to add a unique (host.name) to the metric resource(s), allowing users to filter between multiple agent systems. resourcedetection: detectors: ["system"] system: hostname_sources: ["os"] normalizesums: batch: exporters: chronicle/powershell: endpoint: malachiteingestion-pa.googleapis.com # Adjust the path to the credentials file you downloaded in Step 1 creds: '/path/to/ingestion-authentication-file.json' log_type: 'IIS' override_log_type: false raw_log_field: body customer_id: '<customer_id>' service: pipelines: logs/winpowershell: receivers: - iis processors: - resourcedetection - normalizesums - batch exporters: [chronicle/iis]
Replace
<customer_id>
with the actual customer ID.Update
/path/to/ingestion-authentication-file.json
to the path where the authentication file was saved in the Get Google SecOps ingestion authentication file section.After saving the
config.yaml
file, start theobservIQ Distro for Open Telemetry Collector
Service.
Restart the Bindplane agent to apply the changes
To restart the Bindplane agent in Windows, you can either use the Services console or enter the following command:
net stop BindPlaneAgent && net start BindPlaneAgent
UDM Mapping Table
Log field | UDM mapping | Logic |
---|---|---|
@timestamp | metadata.event_timestamp | The timestamp of the event as recorded in the raw log. |
@version | metadata.product_version | The version of the IIS server. |
AgentDevice | additional.fields.AgentDevice.value.string_value | The device that generated the log. |
AgentLogFile | additional.fields.AgentLogFile.value.string_value | The name of the log file. |
ASP.NET_SessionId | network.session_id | The session ID of the user. |
c-ip | principal.ip | The IP address of the client. |
Channel | security_result.about.resource.attribute.labels.Channel.value | The channel where the event was logged. |
ChannelID | security_result.about.resource.attribute.labels.ChannelID.value | The ID of the channel where the event was logged. |
Computer | target.hostname | The hostname of the target machine. |
cs-bytes | network.received_bytes | The number of bytes received from the client. |
cs-host | principal.hostname, principal.asset.hostname | The hostname of the client. |
cs-method | network.http.method | The HTTP method used by the client. |
cs-uri-query | target.url | The query string of the URL requested by the client. |
cs-uri-stem | target.url | The path of the URL requested by the client. |
cs-username | principal.user.user_display_name | The username of the client. |
cs-version | network.tls.version_protocol | The HTTP version used by the client. |
cs(Cookie) | Used to extract cookie information. | |
cs(Referer) | network.http.referral_url | The URL that referred the client to the current page. |
cs(User-Agent) | network.http.user_agent | The user agent of the client. |
csbyte | network.received_bytes | The number of bytes received from the client. |
cshost | principal.hostname, principal.asset.hostname | The hostname of the client. |
csip | principal.ip, principal.asset.ip | The IP address of the client. |
csmethod | network.http.method | The HTTP method used by the client. |
csreferer | network.http.referral_url | The URL that referred the client to the current page. |
csuseragent | network.http.user_agent | The user agent of the client. |
csusername | principal.user.user_display_name | The username of the client. |
csversion | network.tls.version_protocol | The HTTP version used by the client. |
date | Used to construct the event timestamp if the raw log timestamp is invalid. | |
description | security_result.description | A description of the event. |
devicename | target.hostname | The hostname of the target machine. |
dst_ip | target.ip, target.asset.ip | The IP address of the target machine. |
dst_port | target.port | The port number of the target machine. |
duration | The duration of the request in milliseconds. | |
EventEnqueuedUtcTime | additional.fields.EventEnqueuedUtcTime.value.string_value | The time when the event was enqueued in UTC. |
EventID | metadata.product_log_id | The ID of the event. |
EventProcessedUtcTime | additional.fields.EventProcessedUtcTime.value.string_value | The time when the event was processed in UTC. |
EventTime | metadata.event_timestamp | The timestamp of the event. |
EventType | metadata.product_event_type | The type of the event. |
file_path | target.file.full_path | The full path of the file involved in the event. |
FilterId | security_result.about.resource.attribute.labels.FilterId.value | The ID of the filter. |
FilterKey | security_result.about.resource.attribute.labels.FilterKey.value | The key of the filter. |
FilterName | security_result.about.resource.attribute.labels.FilterName.value | The name of the filter. |
FilterType | security_result.about.resource.attribute.labels.FilterType.value | The type of the filter. |
host | target.hostname | The hostname of the target machine. |
host.architecture | principal.asset.hardware.cpu_platform | The architecture of the host machine. |
host.geo.name | additional.fields.geo_name.value.string_value | The geographical location of the host machine. |
host.hostname | target.hostname, target.asset.hostname | The hostname of the host machine. |
host.id | observer.asset_id | The ID of the host machine. |
host.ip | principal.ip, principal.asset.ip | The IP address of the host machine. |
host.mac | principal.mac | The MAC address of the host machine. |
host.os.build | additional.fields.os_build.value.string_value | The build number of the operating system on the host machine. |
host.os.kernel | principal.platform_patch_level | The kernel version of the operating system on the host machine. |
host.os.name | additional.fields.os_name.value.string_value | The name of the operating system on the host machine. |
host.os.platform | principal.platform | The platform of the operating system on the host machine. |
host.os.version | principal.platform_version | The version of the operating system on the host machine. |
http_method | network.http.method | The HTTP method used by the client. |
http_response | network.http.response_code | The HTTP response code. |
http_status_code | network.http.response_code | The HTTP status code of the response. |
http_substatus | additional.fields.sc_substatus.value.string_value | The HTTP substatus code of the response. |
instance | additional.fields.instance.value.string_value | The instance ID of the task. |
intermediary_devicename | intermediary.hostname, intermediary.asset.hostname | The hostname of the intermediary device. |
json_message | The raw log message in JSON format. | |
kv_fields | Used to extract key-value pairs from the raw log message. | |
LayerKey | security_result.about.resource.attribute.labels.LayerKey.value | The key of the layer. |
LayerName | security_result.about.resource.attribute.labels.LayerName.value | The name of the layer. |
LayerId | security_result.about.resource.attribute.labels.LayerId.value | The ID of the layer. |
log.file.path | target.file.full_path | The full path of the log file. |
log.offset | metadata.product_log_id | The offset of the event in the log file. |
logstash.collect.host | observer.hostname | The hostname of the machine that collected the log. |
logstash.process.host | intermediary.hostname | The hostname of the machine that processed the log. |
logstash_json_message | The raw log message in JSON format. | |
message | security_result.description | The raw log message. |
ministry | additional.fields.ministry.value.string_value | The ministry associated with the event. |
name | The name of the entity. | |
NewValue | additional.fields.NewValue.value.string_value | The new value of the configuration setting. |
OldValue | additional.fields.OldValue.value.string_value | The old value of the configuration setting. |
port | principal.port | The port number of the client. |
priority_code | The priority code of the syslog message. | |
ProcessID | principal.process.pid | The process ID of the process that generated the event. |
ProviderGuid | security_result.about.resource.attribute.labels.ProviderGuid.value | The GUID of the provider. |
ProviderKey | security_result.about.resource.attribute.labels.ProviderKey.value | The key of the provider. |
ProviderName | security_result.about.resource.attribute.labels.ProviderName.value | The name of the provider. |
referrer_url | network.http.referral_url | The URL that referred the client to the current page. |
request_url | target.url | The URL requested by the client. |
s-computername | target.hostname | The hostname of the target machine. |
s-ip | target.ip, target.asset.ip | The IP address of the target machine. |
s-port | target.port | The port number of the target machine. |
s-sitename | additional.fields.sitename.value.string_value | The name of the site. |
sc-bytes | network.sent_bytes | The number of bytes sent to the client. |
sc-status | network.http.response_code | The HTTP status code of the response. |
sc-substatus | additional.fields.sc_substatus.value.string_value | The HTTP substatus code of the response. |
sc-win32-status | The Windows status code of the response. | |
scbyte | network.sent_bytes | The number of bytes sent to the client. |
scstatus | network.http.response_code | The HTTP status code of the response. |
severity | security_result.severity | The severity of the event. |
service.type | additional.fields.service_type.value.string_value | The type of the service. |
sIP | principal.ip, principal.asset.ip | The IP address of the client. |
sPort | principal.port | The port number of the client. |
sSiteName | additional.fields.sitename.value.string_value | The name of the site. |
src_ip | principal.ip, principal.asset.ip, observer.ip | The IP address of the client. |
src_port | principal.port | The port number of the client. |
sysdate | The date and time of the syslog message. | |
syslog_facility | security_result.severity_details | The facility of the syslog message. |
syslog_pri | The priority of the syslog message. | |
syslog_severity | security_result.severity_details | The severity of the syslog message. |
syslog_severity_code | The severity code of the syslog message. | |
tags | security_result.rule_name | Tags associated with the event. |
task | additional.fields.task.value.string_value | The name of the task. |
time | Used to construct the event timestamp if the raw log timestamp is invalid. | |
time-taken | The duration of the request in milliseconds. | |
uri_query | target.url | The query string of the URL requested by the client. |
user_agent | network.http.user_agent | The user agent of the client. |
UserName | target.user.userid | The username of the user. |
UserSid | target.user.windows_sid | The Windows SID of the user. |
Weight | security_result.about.resource.attribute.labels.Weight.value | The weight of the filter. |
win32_status | The Windows status code of the response. | |
xforwardedfor | The X-Forwarded-For header, containing a comma-separated list of IP addresses. | |
metadata.log_type | IIS |
|
network.direction | INBOUND |
|
metadata.vendor_name | Microsoft |
|
metadata.product_name | Internet Information Server |
|
metadata.event_type | NETWORK_HTTP , USER_UNCATEGORIZED , GENERIC_EVENT , STATUS_UPDATE , USER_LOGOUT , USER_LOGIN |
|
extensions.auth.type | MACHINE |
Need more help? Get answers from Community members and Google SecOps professionals.