Collect Microsoft Defender for Identity logs

This document explains how to ingest the Microsoft Defender for Identity logs to Google Security Operations using Azure Storage. The parser processes JSON logs, or CEF formatted logs if the JSON parsing fails. It extracts fields, performs data transformations such as string conversions, renaming, and merging, and maps them to the Unified Data Model (UDM), handling various log formats and enriching the data with additional context like labels and authentication details.

Before you begin

Ensure that you have the following prerequisites:

• Google SecOps instance
• active Azure tenant
• Privileged access to Azure and Administrative Security role

Configure Azure Storage account

1. In the Azure console, search for Storage accounts.
2. Click Create.
3. Specify values for the following input parameters:
   • Subscription: select the subscription.
   • Resource Group: select the resource group.
   • Region: select the region.
   • Performance: select the type of performance (Standard recommended).
   • Redundancy: select the type of redundancy (GRS or LRS recommended).
   • Storage account name: enter a name for the new Storage account.
4. Click Review + create.
5. Review the overview of the account and click Create.
6. From the storage account Overview page, select submenu Access keys in Security + networking.
7. Click Show next to key1 or key2.
8. Click Copy to clipboard to copy the key.
9. Save the key in a secure location for future reference.
10. From the storage account Overview page, select submenu Endpoints in Settings.
11. Click Copy to clipboard to copy the Blob service endpoint URL. (for example, https://<storageaccountname>.blob.core.windows.net).
12. Save the endpoint URL in a secure location for future reference.
13. Go to Overview > JSON View.
14. Copy and save the Storage Resource ID.

Configure Log Export for Microsoft Defender for Identity

1. Sign in to the Defender Portal using a privileged account.
2. Go to Settings.
3. Select the Microsoft Defender XDR tab.
4. Select Streaming API from the general section and click Add.
5. Select Forward events to Azure Storage.
6. Provide the following configuration details:
   • Name: Enter a unique and meaningful name.
   • Select Forward events to Azure Storage.
   • Storage Account Resource ID: Enter the Azure Storage resource ID copied earlier.
   • Event Types: Select both Alerts & Behaviors and Devices.
7. Click Submit.

Configure a feed in Google SecOps to ingest the Defender for Identity logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, Defender Identity Logs).
4. Select Microsoft Azure Blob Storage as the Source type.
5. Select Microsoft Defender for Identity as the Log type.
6. Click Next.

7. Specify values for the following input parameters:

   • Azure uri: the blob endpoint URL.

     ENDPOINT_URL/BLOB_NAME

     Replace the following:

     • ENDPOINT_URL: the blob endpoint URL. (https://<storageaccountname>.blob.core.windows.net)
     • BLOB_NAME: the name of the blob. (such as, insights-logs-<logname>)

   • URI is a: select according to log stream configuration (Single file | Directory | Directory which includes subdirectories).

   • Source deletion options: select deletion option according to your preference.

   • Shared key: the access key to the Azure Blob Storage.
   • Asset namespace: the asset namespace.
   • Ingestion labels: the label to be applied to the events from this feed.

8. Click Next.

9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM mapping table

Log Field	UDM Mapping	Logic
category	metadata.log_type	The raw log category field is mapped to metadata.log_type.
properties.AccountDisplayName	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AccountName	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AccountUpn	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.ActionType	metadata.product_event_type	The raw log properties.ActionType field is mapped to metadata.product_event_type.
properties.AdditionalFields.ACTOR.ACCOUNT	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.ACTOR.DEVICE	principal.asset.asset_id	The parser extracts the value of properties.AdditionalFields.ACTOR.DEVICE and prepends ASSET ID:.
properties.AdditionalFields.ACTOR.ENTITY_USER	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.Count	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.DestinationComputerDnsName	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.DestinationComputerObjectGuid	target.asset.product_object_id	The first element of the array properties.AdditionalFields.DestinationComputerObjectGuid is mapped to target.asset.product_object_id. Subsequent elements are mapped to additional.fields with keys like DestinationComputerObjectGuid_1, DestinationComputerObjectGuid_2, etc.
properties.AdditionalFields.DestinationComputerOperatingSystem	target.asset.platform_software.platform_version	The first element of the array properties.AdditionalFields.DestinationComputerOperatingSystem is mapped to target.asset.platform_software.platform_version. Subsequent elements are mapped to additional.fields with keys like DestinationComputerOperatingSystem_1, DestinationComputerOperatingSystem_2, etc.
properties.AdditionalFields.DestinationComputerOperatingSystemType	target.asset.platform_software.platform	If the value is windows, the UDM field is set to WINDOWS.
properties.AdditionalFields.DestinationComputerOperatingSystemVersion	target.platform_version	The first element of the array properties.AdditionalFields.DestinationComputerOperatingSystemVersion is mapped to target.platform_version. Subsequent elements are mapped to additional.fields with keys like DestinationComputerOperatingSystemVersion1, DestinationComputerOperatingSystemVersion2, etc.
properties.AdditionalFields.FROM.DEVICE	principal.asset.asset_id	The parser extracts the value of properties.AdditionalFields.FROM.DEVICE and prepends ASSET ID:.
properties.AdditionalFields.KerberosDelegationType	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.SourceAccountId	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.SourceAccountSid	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.SourceComputerObjectGuid	principal.asset.product_object_id	The raw log properties.AdditionalFields.SourceComputerObjectGuid field is mapped to principal.asset.product_object_id.
properties.AdditionalFields.SourceComputerOperatingSystem	principal.asset.platform_software.platform_version	The raw log properties.AdditionalFields.SourceComputerOperatingSystem field is mapped to principal.asset.platform_software.platform_version.
properties.AdditionalFields.SourceComputerOperatingSystemType	principal.asset.platform_software.platform_version	If the value is windows, the UDM field is set to WINDOWS.
properties.AdditionalFields.SourceComputerOperatingSystemVersion	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.Spns	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.TARGET_OBJECT.ENTITY_USER	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.AdditionalFields.TARGET_OBJECT.USER	target.user.userid	The first element of the array properties.AdditionalFields.TARGET_OBJECT.USER is mapped to target.user.userid. Subsequent elements are mapped to additional.fields with keys like TARGET_OBJECT.USER_1, TARGET_OBJECT.USER_2, etc.
properties.AdditionalFields.TO.DEVICE	target.asset.asset_id	The first element of the array properties.AdditionalFields.TO.DEVICE is mapped to target.asset.asset_id with ASSET ID: prepended. Subsequent elements are mapped to additional.fields with keys like TODEVICE1, TODEVICE2, etc.
properties.AuthenticationDetails	extensions.auth.auth_details	The parser removes curly braces, square brackets, and double quotes from the value and prepends AuthenticationDetails:.
properties.DeliveryAction	additional.fields	Mapped with key DeliveryAction.
properties.DeliveryLocation	additional.fields	Mapped with key DeliveryLocation.
properties.DestinationDeviceName	target.hostname, target.asset.hostname	The raw log properties.DestinationDeviceName field is mapped to both target.hostname and target.asset.hostname.
properties.DestinationIPAddress	target.ip, target.asset.ip	The raw log properties.DestinationIPAddress field is mapped to both target.ip and target.asset.ip.
properties.DestinationPort	target.port	The raw log properties.DestinationPort field is mapped to target.port.
properties.DeviceName	principal.hostname, principal.asset.hostname	The raw log properties.DeviceName field is mapped to both principal.hostname and principal.asset.hostname.
properties.EmailClusterId	additional.fields	Mapped with key EmailClusterId.
properties.EmailDirection	network.direction	If the value is Inbound, the UDM field is set to INBOUND. If the value is Outbound, the UDM field is set to OUTBOUND. Otherwise, it's set to UNKNOWN_DIRECTION.
properties.EmailLanguage	additional.fields	Mapped with key EmailLanguage.
properties.InitiatingProcessAccountDomain	principal.administrative_domain	The raw log properties.InitiatingProcessAccountDomain field is mapped to principal.administrative_domain.
properties.InitiatingProcessAccountSid	principal.user.windows_sid	The raw log properties.InitiatingProcessAccountSid field is mapped to principal.user.windows_sid.
properties.InitiatingProcessCommandLine	principal.process.command_line	The raw log properties.InitiatingProcessCommandLine field is mapped to principal.process.command_line.
properties.InitiatingProcessFileName	principal.process.file.full_path	Used in combination with properties.InitiatingProcessFolderPath to construct the full path. If properties.InitiatingProcessFolderPath already contains the filename, it's used directly.
properties.InitiatingProcessFolderPath	principal.process.file.full_path	Used in combination with properties.InitiatingProcessFileName to construct the full path.
properties.InitiatingProcessId	principal.process.pid	The raw log properties.InitiatingProcessId field is mapped to principal.process.pid.
properties.InitiatingProcessIntegrityLevel	about.labels	Mapped with key InitiatingProcessIntegrityLevel.
properties.InitiatingProcessMD5	principal.process.file.md5	The raw log properties.InitiatingProcessMD5 field is mapped to principal.process.file.md5.
properties.InitiatingProcessParentId	principal.process.parent_process.pid	The raw log properties.InitiatingProcessParentId field is mapped to principal.process.parent_process.pid.
properties.InitiatingProcessParentFileName	principal.process.parent_process.file.full_path	The raw log properties.InitiatingProcessParentFileName field is mapped to principal.process.parent_process.file.full_path.
properties.InitiatingProcessSHA1	principal.process.file.sha1	The raw log properties.InitiatingProcessSHA1 field is mapped to principal.process.file.sha1.
properties.InitiatingProcessSHA256	principal.process.file.sha256	The raw log properties.InitiatingProcessSHA256 field is mapped to principal.process.file.sha256.
properties.InitiatingProcessTokenElevation	about.labels	Mapped with key InitiatingProcessTokenElevation.
properties.InternetMessageId	additional.fields	The parser removes angle brackets and maps the value with key InternetMessageId.
properties.IPAddress	principal.ip, principal.asset.ip	The raw log properties.IPAddress field is mapped to both principal.ip and principal.asset.ip.
properties.LogonType	extensions.auth.mechanism	Used to derive the value for extensions.auth.mechanism.
properties.Port	principal.port	The raw log properties.Port field is mapped to principal.port.
properties.PreviousRegistryKey	src.registry.registry_key	The raw log properties.PreviousRegistryKey field is mapped to src.registry.registry_key.
properties.PreviousRegistryValueData	src.registry.registry_value_data	The raw log properties.PreviousRegistryValueData field is mapped to src.registry.registry_value_data.
properties.PreviousRegistryValueName	src.registry.registry_value_name	The raw log properties.PreviousRegistryValueName field is mapped to src.registry.registry_value_name.
properties.Query	principal.user.attribute.labels	Mapped with key LDAP Search Scope.
properties.RecipientEmailAddress	Not Mapped	This field is not mapped to the IDM object in the UDM.
properties.RegistryKey	target.registry.registry_key	The raw log properties.RegistryKey field is mapped to target.registry.registry_key.
properties.RegistryValueData	target.registry.registry_value_data	The raw log properties.RegistryValueData field is mapped to target.registry.registry_value_data.
properties.RegistryValueName	target.registry.registry_value_name	The raw log properties.RegistryValueName field is mapped to target.registry.registry_value_name.
properties.ReportId	about.labels	Mapped with key ReportId.
properties.SenderIPv4	principal.ip, principal.asset.ip	The raw log properties.SenderIPv4 field is mapped to both principal.ip and principal.asset.ip.
properties.SenderMailFromAddress	principal.user.attribute.labels	Mapped with key SenderMailFromAddress.
properties.SenderMailFromDomain	principal.user.attribute.labels	Mapped with key SenderMailFromDomain.
properties.SenderObjectId	principal.user.product_object_id	The raw log properties.SenderObjectId field is mapped to principal.user.product_object_id.
properties.Timestamp	metadata.event_timestamp	The raw log properties.Timestamp field is mapped to metadata.event_timestamp.
tenantId	observer.cloud.project.id	The raw log tenantId field is mapped to observer.cloud.project.id.
N/A	extensions.auth.type	The value MACHINE is assigned by the parser.
N/A	metadata.event_type	Derived based on the category and properties.ActionType fields. Can be USER_LOGIN, USER_RESOURCE_ACCESS, USER_CHANGE_PASSWORD, REGISTRY_MODIFICATION, REGISTRY_DELETION, REGISTRY_CREATION, GENERIC_EVENT, or STATUS_UPDATE.
N/A	metadata.vendor_name	The value Microsoft is assigned by the parser.
N/A	metadata.product_name	The value Microsoft Defender Identity is assigned by the parser.
cs1	metadata.url_back_to_product	The raw log cs1 field is mapped to metadata.url_back_to_product.
externalId	metadata.product_log_id	The raw log externalId field is mapped to metadata.product_log_id.
msg	metadata.description	The raw log msg field is mapped to metadata.description.
rule_name	security_result.rule_name	The raw log rule_name field is mapped to security_result.rule_name.
severity	security_result.severity	The raw log severity field is mapped to security_result.severity.
shost	principal.hostname, principal.asset.hostname	The raw log shost field is mapped to both principal.hostname and principal.asset.hostname.
src	principal.ip	The raw log src field is mapped to principal.ip.
suser	principal.user.user_display_name	The raw log suser field is mapped to principal.user.user_display_name.
time	metadata.event_timestamp	The raw log time field is mapped to metadata.event_timestamp.
userid	principal.user.userid	The raw log userid field is mapped to principal.user.userid.
N/A	security_result.action	Derived based on the properties.ActionType field. Can be ALLOW or BLOCK.
N/A	security_result.summary	Derived from either the category field or the properties.ActionType field.

Changes

2025-02-11

Enhancement:

• Added support to parse unparsed fields.

2024-10-14

Enhancement:

• Mapped device_event_class_id to security_result.rule_name.
• Mapped event_name to security_result.description.
• Mapped app and alert_id to additional.fields.
• Mapped externalId to metadata.product_log_id.

2024-09-12

Enhancement:

• Added support for array-type logs.

2024-08-09

Enhancement:

• Added support to parse syslog logs.

2024-06-25

Enhancement:

• Added support to parse unparsed logs.
• If properties.AdditionalFields.TARGET_OBJECT.USER is absent, mapped properties.AccountName to target.user.userid.
• If properties.AccountName and properties.AdditionalFields.PRINCIPAL_OBJECT.USER are absent, mapped properties.AccountDisplayName to principal.user.userid.
• Mapped properties.Location to principal.location.country_or_region.
• Mapped properties.AccountObjectId to principal.user.attribute.labels.

2024-04-15

Enhancement:

• Added support to map DestinationComputerObjectGuid, DestinationComputerOperatingSystem, DestinationComputerOperatingSystemVersion, TO.DEVICE fields, when the value of these fields is a list instead of a string.

2022-07-27

Enhancement:

• Mapped metadata.event_type to REGISTRY_MODIFICATION where properties.ActionType is not null.
• Mapped metadata.event_type to REGISTRY_DELETION where properties.ActionType is RegistryValueDeleted.
• Mapped metadata.event_type to REGISTRY_CREATION where properties.ActionType is RegistryValueCreated.
• Mapped properties.InitiatingProcessFolderPath to process.file.full_path.
• Mapped about.labels to properties.InitiatingProcessIntegrityLevel.
• Mapped properties.DeviceId to principal.asset_id.
• Mapped properties.InitiatingProcessTokenElevation to about.labels.
• Mapped properties.InitiatingProcessParentFileName to principal.process.parent_process.file.full_path.
• Mapped properties.InitiatingProcessMD5 to principal.process.file.md5.
• Mapped properties.InitiatingProcessSHA256 to principal.process.file.sha256.
• Mapped properties.InitiatingProcessSHA1 to principal.process.file.sha1.
• Mapped properties.InitiatingProcessId to principal.process.pid.
• Mapped properties.InitiatingProcessCommandLine to principal.process.command_line.
• Mapped properties.InitiatingProcessAccountSid to principal.user.windows_sid.
• Mapped properties.InitiatingProcessAccountDomain to principal.administrative_domain.
• Mapped properties.RegistryKey to target.registry.registry_key.
• Mapped properties.RegistryValueName to target.registry.registry_value_name.
• Mapped properties.RegistryValueData to target.registry.registry_value_data.
• Mapped properties.PreviousRegistryKey to src.registry.registry_key.
• Mapped properties.PreviousRegistryValueName to src.registry.registry_value_name.
• Mapped properties.PreviousRegistryValueData to src.registry.registry_value_data.

2022-04-22

Enhancement:

• Newly created parser.

Need more help? Get answers from Community members and Google SecOps professionals.