Collect Akamai Cloud Monitor logs

This document explains how to ingest Akamai Cloud Monitor (Load Balancer, Traffic Shaper, ADC) logs to Google Security Operations using AWS S3. Akamai pushes JSON events to your HTTPS endpoint; an API Gateway + Lambda receiver writes the events to S3 (JSONL, gz). The parser transforms the JSON logs into UDM. It extracts fields from the JSON payload, performs data type conversions, renames fields to match the UDM schema, and handles specific logic for custom fields and URL construction. It also incorporates error handling and conditional logic based on field presence.

Before you begin

Make sure you have the following prerequisites:

• Google SecOps instance
• Privileged access to Akamai Control Center and Property Manager
• Privileged access to AWS*(S3, IAM, Lambda, API Gateway)

Configure AWS S3 bucket and IAM for Google SecOps

1. Create Amazon S3 bucket following this user guide: Creating a bucket
2. Save bucket Name and Region for future reference (for example, akamai-cloud-monitor).
3. Create a user following this user guide: Creating an IAM user.
4. Select the created User.
5. Select the Security credentials tab.
6. Click Create Access Key in the Access Keys section.
7. Select Third-party service as the Use case.
8. Click Next.
9. Optional: add a description tag.
10. Click Create access key.
11. Click Download CSV file to save the Access Key and Secret Access Key for later use.
12. Click Done.
13. Select the Permissions tab.
14. Click Add permissions in the Permissions policies section.
15. Select Add permissions.
16. Select Attach policies directly
17. Search for and select the AmazonS3FullAccess policy.
18. Click Next.
19. Click Add permissions.

Configure the IAM policy and role for S3 uploads (Lambda)

1. In the AWS Console, go to IAM > Policies > Create policy > JSON and paste the policy below.

2. JSON Policy (replace akamai-cloud-monitor with your S3 bucket name):

   {
   "Version": "2012-10-17",
   "Statement": [
       {
       "Sid": "AllowPutAkamaiObjects",
       "Effect": "Allow",
       "Action": ["s3:PutObject"],
       "Resource": "arn:aws:s3:::akamai-cloud-monitor/*"
       }
   ]
   }
   

3. Click Next > Create policy.

4. Go to IAM > Roles > Create role > AWS service > Lambda.

5. Attach the JSON policy.

6. Name the role WriteAkamaiCMToS3Role and click Create role.

Create the Lambda function

1. After the function is created, open the Code tab, delete the stub and enter the following code (akamai_cloud_monitor_to_s3.py):

   #!/usr/bin/env python3
   # Lambda: Receive Akamai Cloud Monitor POST, write JSONL (gz) to S3
   
   import os, json, gzip, io, uuid, base64, datetime as dt
   import boto3
   
   S3_BUCKET  = os.environ["S3_BUCKET_NAME"]
   S3_PREFIX  = os.environ.get("S3_PREFIX", "akamai/cloud-monitor/json/").strip("/") + "/"
   INGEST_TOKEN = os.environ.get("INGEST_TOKEN")  # optional shared secret in URL query (?token=...)
   
   s3 = boto3.client("s3")
   
   def _write_jsonl_gz(objs: list) -> str:
       key = f"{dt.datetime.utcnow():%Y/%m/%d}/akamai-cloud-monitor-{uuid.uuid4()}.json.gz"
       buf = io.BytesIO()
       with gzip.GzipFile(fileobj=buf, mode="w") as gz:
           for o in objs:
               gz.write((json.dumps(o, separators=(",", ":")) + "n").encode())
       buf.seek(0)
       s3.upload_fileobj(
           buf,
           S3_BUCKET,
           f"{S3_PREFIX}{key}",
           ExtraArgs={
               "ContentType": "application/json",
               "ContentEncoding": "gzip",
           },
       )
       return f"s3://{S3_BUCKET}/{S3_PREFIX}{key}"
   
   def _parse_records_from_event(event) -> list:
       # HTTP API (Lambda proxy) event: body is a JSON string
       body = event.get("body") or ""
       if event.get("isBase64Encoded"):
           body = base64.b64decode(body).decode("utf-8", "replace")
       try:
           data = json.loads(body)
       except Exception:
           # accept line-delimited JSON as pass-through
           try:
               return [json.loads(line) for line in body.splitlines() if line.strip()]
           except Exception:
               return []
       if isinstance(data, list):
           return data
       if isinstance(data, dict):
           return [data]
       return []
   
   def lambda_handler(event, context=None):
       # Optional shared-secret verification via query parameter (?token=...)
       if INGEST_TOKEN:
           qs = event.get("queryStringParameters") or {}
           token = qs.get("token")
           if token != INGEST_TOKEN:
               return {"statusCode": 403, "body": "forbidden"}
   
       records = _parse_records_from_event(event)
       if not records:
           return {"statusCode": 204, "body": "no content"}
   
       key = _write_jsonl_gz(records)
       return {
           "statusCode": 200,
           "headers": {"Content-Type": "application/json"},
           "body": json.dumps({"ok": True, "s3_key": key, "count": len(records)}),
       }
   

2. Go to Configuration > Environment variables > Edit.

3. Click Add new environment variable and set the following values:

   Environment variables

   Key	Example
   S3_BUCKET_NAME	akamai-cloud-monitor
   S3_PREFIX	akamai/cloud-monitor/json/
   INGEST_TOKEN	random-shared-secret

4. Go to Configuration > General configuration.

5. Click Edit and set Timeout to 5 minutes (300 seconds).

6. Click Save.

Create Amazon API Gateway (HTTPS endpoint for Akamai)

1. In the AWS Console, go to API Gateway > Create API.
2. Select HTTP API > Build.
3. Provide the following configuration details:
   • Integrations: Choose Lambda and select akamai_cloud_monitor_to_s3.
   • Routes: Add ANY /{proxy+} or create a specific route (for example, POST /akamai/cloud-monitor).
   • Stages: Create or use $default.
4. Deploy the API and copy the Invoke URL (for example, https://abc123.execute-api.<region>.amazonaws.com).

Configure Akamai Cloud Monitor to push logs

1. In Akamai Control Center, open your Property in Property Manager.
2. Click Add Rule > choose Cloud Management.
3. Add Cloud Monitor Instrumentation and select required Datasets.
4. Add Cloud Monitor Data Delivery.
   • Delivery Hostname: enter your API Gateway Invoke URL (for example, abc123.execute-api.<region>.amazonaws.com).
   • Delivery URL Path: your route plus an optional query token, for example: /akamai/cloud-monitor?token=<INGEST_TOKEN>.
5. Save and Activate the property version.

Configure a feed in Google SecOps to ingest Akamai Cloud Monitor (S3 JSON)

1. Go to SIEM Settings > Feeds.
2. Click Add New Feed.
3. In the Feed name field, enter a name for the feed (for example, Akamai Cloud Monitor — S3).
4. Select Amazon S3 V2 as the Source type.
5. Select Akamai Cloud Monitor as the Log type.
6. Click Next.
7. Specify values for the following input parameters:
   • S3 URI: s3://akamai-cloud-monitor/akamai/cloud-monitor/json/
   • Source deletion options: Whether to delete files and/or directories after transferring.
   • Maximum File Age: Includes files modified in the last number of days. Default is 180 days.
   • Access Key ID: A 20-character alphanumeric account access key (e.g., AKIAIOSFODNN7EXAMPLE).
   • Secret Access Key: A 40-character alphanumeric account secret access key (e.g., wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY).
   • Asset namespace: akamai.cloud_monitor
   • Ingestion labels: Labels are added to all the events from this feed (for example, source=akamai_cloud_monitor, format=json).
8. Click Next.
9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM Mapping Table

Need more help? Get answers from Community members and Google SecOps professionals.