Collect Qualys Virtual Scanner logs

This parser transforms raw JSON formatted Qualys Virtual Scanner logs into a structured format conforming to the Google Security Operations UDM. It extracts relevant fields like asset information, scan details, and detected vulnerabilities, mapping them to corresponding UDM fields for consistent representation and analysis.

Before you begin

• Ensure that you have a Google Security Operations instance.
• Ensure that you have privileged access to Google Cloud.
• Ensure that you have privileged access to Qualys.

Enable Required APIs:

1. Sign in to the Google Cloud console.
2. Go to APIs & Services > Library.
3. Search for the following APIs and enable them:
   • Cloud Functions API
   • Cloud Scheduler API
   • Cloud Pub/Sub (required for Cloud Scheduler to invoke functions)

Create a Google Cloud Storage Bucket

1. Sign in to the Google Cloud console.

2. Go to the Cloud Storage Buckets page.

   Go to Buckets

3. Click Create.

4. Configure the bucket:

   • Name: enter a unique name that meets the bucket name requirements (for example, qualys-vscanner-bucket).
   • Choose where to store your data: select a location.
   • Choose a storage class for your data: either select a default storage class for the bucket, or select Autoclass for automatic storage class management.
   • Choose how to control access to objects: select not to enforce public access prevention, and select an access control model for your bucket's objects.
   • Storage class: Choose based on your needs (for example, Standard).

5. Click Create.

Create a Google Cloud Service Account

1. Go to to IAM & Admin > Service Accounts.
2. Create a new service account.
3. Give it a descriptive name (for example, qualys-user).
4. Grant the service account with Storage Object Admin role on the Cloud Storage bucket you created in the previous step.
5. Grant the service account with Cloud Functions Invoker role.
6. Create an SSH key for the service account.
7. Download a JSON key file for the service account. Keep this file secure.

Optional: Create a dedicated API User in Qualys

1. Sign in to the Qualys console.
2. Go to Users.
3. Click New > User.
4. Enter the General Information required for the user.
5. Select User Role tab.
6. Make sure the role has the API Access checkbox selected.
7. Click Save.

Identify your specific Qualys API URL

Option 1

Identify your URLs as mentioned in the platform identification.

Option 2

1. Sign in to the Qualys console.
2. Go to Help > About.
3. Scroll to see this information under Security Operations Center (SOC).
4. Copy the Qualys API URL.

Configure the Cloud Function

1. Go to Cloud Functions in the Google Cloud console.
2. Click Create Function.

3. Configure the Function:

   • Name: enter a name for your function (for example, fetch-qualys-vscanner).
   • Region: select a region close to your Bucket.
   • Trigger: choose HTTP trigger if needed or Cloud Pub/Sub for scheduled execution.
   • Authentication: secure with authentication.
   • Write the Code with an inline editor:

   ```python
   from google.cloud import storage
   import requests
   import base64
   import json
   
   # Google Cloud Storage Configuration
   BUCKET_NAME = "<bucket-name>"
   FILE_NAME = "qualys_virtual_scanners.json"
   
   # Qualys API Credentials
   QUALYS_USERNAME = "qualys-username"
   QUALYS_PASSWORD = "<qualys-password>"
   QUALYS_BASE_URL = "https://<qualys_base_url>"  # for example, https://qualysapi.qualys.com
   
   def fetch_virtual_scanners():
       """Fetch Virtual Scanner details from Qualys."""
       auth = base64.b64encode(f"{QUALYS_USERNAME}:{QUALYS_PASSWORD}".encode()).decode()
       headers = {
           "Authorization": f"Basic {auth}",
           "Content-Type": "application/xml"
       }
       url = f"{QUALYS_BASE_URL}/api/2.0/fo/scanner/"
       payload = {
           "action": "list",
           "scanner_type": "virtual"
       }
       response = requests.post(url, headers=headers, data=payload)
       response.raise_for_status()
       return response.text  # Qualys API returns XML data
   
   def upload_to_gcs(data):
       """Upload data to Google Cloud Storage."""
       client = storage.Client()
       bucket = client.get_bucket(BUCKET_NAME)
       blob = bucket.blob(FILE_NAME)
       blob.upload_from_string(data, content_type="application/xml")
   
   def main(request):
       """Cloud Function entry point."""
       try:
           scanners = fetch_virtual_scanners()
           upload_to_gcs(scanners)
           return "Qualys Virtual Scanners data uploaded to Cloud Storage successfully!"
       except Exception as e:
           return f"An error occurred: {e}", 500
   ```
   

4. Click Deploy after completing the configuration.

Configure Cloud Scheduler

1. Go to Cloud Scheduler in the Google Cloud console.
2. Click Create Job.

3. Configure the job:

   • Name: enter a name for your job (for example, trigger-fetch-qualys-vscanner).
   • Frequency: use cron syntax to specify the schedule (for example, 0 0 * * * for daily at midnight).
   • Time Zone: set your preferred time zone.
   • Trigger Type: Choose HTTP.
   • Trigger URL: enter the Cloud Function's URL (found in the function details after deployment).
   • Method: Choose POST.

4. Create the Job.

Configure a feed in Google SecOps to ingest Qualys Virtual Scanner logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, Qualys Virtual Scanner Logs).
4. Select Google Cloud Storage as the Source type.
5. Select Qualys Virtual Scanner as the Log type.
6. Click Next.

7. Specify values for the following input parameters:

   • Storage Bucket URI: the Google Cloud storage bucket source URI.
   • URI is a: select Single file.
   • Source deletion option: select the deletion option according to your preference.
   • Asset namespace: the asset namespace.
   • Ingestion labels: the label to be applied to the events from this feed.

8. Click Next.

9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM Mapping Table

Changes

2023-08-21

• Mapped the "detection.UNIQUE_VULN_ID" value from the original log to the "threat.detection_fields" field in the UDM.

2023-07-31

• Newly created parser.