Change log for OBSERVEIT

Date Changes
2025-07-17 Enhancement:
- Added a grok pattern to parse the raw log.
- Modified gsub on "kv_data" to replace "([a-zA-Z0-9_-]+)=" with "#$1="
- Added a gsub to replace "type" with "Type" , ", " with " " ,
- event.idm.read_only_udm.principal.process.file.sha256: Newly mapped "sha256" raw log field to "event.idm.read_only_udm.principal.process.file.sha256"
- event.idm.read_only_udm.target.resource.product_object_id: Newly mapped "id" raw log field to "event.idm.read_only_udm.target.resource.product_object_id".
- event.idm.read_only_udm.principal.process.file.size: Newly mapped "size" raw log field to "event.idm.read_only_udm.principal.process.file.size".
- event.idm.read_only_udm.principal.file.file_type: Newly mapped "Type" raw log field to "event.idm.read_only_udm.principal.file.file_type".
- event.idm.read_only_udm.security_result.priority_details: Newly mapped "pri" raw log field to "event.idm.read_only_udm.security_result.priority_details".
- Added a grok pattern to extract "tar_host" and "tar_ip" from "relay raw log field.
- event.idm.read_only_udm.target.hostname,event.idm.read_only_udm.target.asset.hostname: Newly mapped "tar_host" raw log field to "event.idm.read_only_udm.target.hostname", "event.idm.read_only_udm.target.asset.hostname".
- event.idm.read_only_udm.target.ip,event.idm.read_only_udm.target.asset.ip: Newly mapped "tar_ip" raw log field to event.idm.read_only_udm.target.ip", "event.idm.read_only_udm.target.asset.ip".
- event.idm.read_only_udm.security_result.summary: Newly mapped "stat" raw log field to "event.idm.read_only_udm.security_result.summary".
- Modified already existing mapping of "proto" to map "event.idm.read_only_udm.network.application_protocol" correctly.
- Added a regex conditional check before already existing of "to", "from","rcpt".
- event.idm.read_only_udm.security_result.detection_fields: Newly mapped "corrupted", "protected", "duration", "mailer", "tls_verify", and "dsn" raw log fields to event.idm.read_only_udm.security_result.detection_fields.
- event.idm.read_only_udm.additional.fields: Newly mapped "m", "omime", "oext", "lang", "url_count", "virtual", "a", "delay", and "xdelay" raw log fields to event.idm.read_only_udm.additional.fields.
2024-12-13 Enhancement:
- Mapped "_derivatives.direction.source.name" to "target.resource.attribute.labels".
2024-12-09 Enhancement:
-Changed mapping of "reason.name" from "security_result.detection_fields" to "security_result.description".
2024-11-21 Enhancement:
Mapped "resource.target" to "target.resource.attribute.labels".
- Mapped"resource.classification.labels" to "security_result.detection_fields"
- Mapped "partitionKey" to "security_result.detection_fields"
- Mapped "fqid" to "security_result.detection_fields"
- Mapped "context.contextId" to "principal.labels"
- Mapped "context.partitionKey" to "principal.labels"
- Mapped "entity" to "security_result.detection_fields"
- Mapped "feed.instance" to "principal.asset.product_object_id"
- Mapped "incident.reasons" to "security_result.detection_fields"
- Mapped "recipient.id" to "target.user.userid"
- Mapped "recipient.kind" to "target.user.role_description"
- Mapped "recipient.email" to "target.user.email_addresses"
- Mapped "esUrl" to "metadata.url_back_to_product"
- Mapped "policyRoutes" to "security_result.detection_fields"
- Mapped "organization.tenant" to "security_result.detection_fields"
2024-10-17 Enhancement:
- Modified the mapping of "additional.fields" for "value.verticals.key".
- Mapped "remote.host.ip.address" to "principal.ip".
2023-12-15 Enhancement:
- Added support for CEF format logs.
2023-11-03 Enhancement:
- Mapped the fields in "processing.actions" to "security_result.detection_fields".
- Mapped the fields in "organization.customer" to "additional fields".
- Mapped the fields in "organization.instances" to "target.resource.attribute.labels".
- Mapped the fields in "_sys.processing.modules" to "target.resource.attribute.labels".
- Mapped the fields in "_sys.processing.rule.artifacts" to "target.resource.attribute.labels".
- Mapped the fields in "event" to "additional fields".
- Mapped the fields in "activity" to "additional fields".
- Mapped the fields in "endpoint.os" to "additional fields".
- Mapped the fields in "ui.windows.os" to "target.resource.attribute.labels".
- Mapped the "_sys.operation" to "additional fields".
- Mapped "ttl" to "network.dns.answer".
- Mapped "site.url" to "target.url".
- Mapped "site.port" to "target.port".
- Mapped "site.host" to "target.hostname".
- Mapped "site.scheme" to "network.application_protocol".
- Mapped the fields in "site.resource" to "target.resource.attribute.labels".
- Mapped "activity.primaryCategory" to "metadata.product_event_type".
2023-07-28 Enhancement:
- Mapped "feed.region" to "principal.asset.location.country_or_region" from "entity.asset.location.country_or_region".
- Mapped "feed.connection.source.ip" to "principal.asset.ip" from "entity.asset.ip".
- Mapped "feed.id" to "principal.asset.asset_id" from "entity.asset.hostname".
- Mapped "feed.instance" to "principal.asset.product_object_id" from "entity.asset.product_object_id".
- Mapped "principal.asset.category" to "WORKSTATION" when "feed.realm" contains "WORKSTATION".
- Mapped "principal.asset.type" to "WORKSTATION" when "feed.realm" contains "WORKSTATION".
2023-07-21 Enhancement:
- Modified the logic to fetch the file related information from the JSON array instead of always fetching from the first element of the array.
2023-05-08 Bug-fix:
- Mapped "observedAt" to "metadata.event_timestamp".
2023-01-21 Enhancement:
- Mapped "session.id" to "network.session_id".
- Mapped "endpoint.location.geo.coordinates.lon.double" to "target.location.region_longitude".
- Mapped "endpoint.location.geo.coordinates.lat.double" to "target.location.region_latitude".
- Mapped "agent.version" to "metadata.product_version".
- Mapped "agent.kind" to "additional.fields".
- Mapped "context.createdAt" to "metadata.collected_timestamp".
- Mapped "context.sortKey" to "security_result.detection_fields".
- Mapped "user.name" to "principal.user.userid".
- Mapped "resources.0.size.int" to "principal.process.file.size".
- Mapped "host" to "principal.hostname".
- Added conditional check for "time", "proc", "device", and "pid".