Gitsync

GitSync is a robust integration built by Google Security Operations SOAR Professional Services team and designed to synchronize Google Security Operations SOAR components with a git repository. It uses git's internal operations to write directly to the repository itself, essentially making it act as a file storage service. It provides methods to perform the following:

• Migrate Assets between Google Security Operations SOAR instances

• Backup Google Security Operations SOAR assets

• Automatic Documentation

• Creates a "store" to share assets/knowledge

• Version Control

The integration consists of multiple Google Security Operations SOAR jobs - push and pull jobs for every supported asset, and push/pull jobs for the entire Google Security Operations SOAR instance. These jobs don't need to run periodically as they were built to run manually from the IDE, but they can be used as regular jobs (For example, uploading a daily commit). 

GitSync will use Google Security Operations SOAR API to fetch the relevant asset, such as an integration or a visual family, and parse all the available information from that asset (This information will later be rendered to a README.md file that is usually displayed when browsing the repository). It then writes the asset JSON definition and the rendered README to the local repository and pushes it to the remote repository.

Another usage of GitSync is knowledge sharing. Using this integration, a git repository can act as a "store" for assets such as playbooks or ontology settings that were designed before and leverages Google Security Operations SOAR best practices to push the platform to its best.

Prerequisites

Pushing/Pulling an existing repository:

• Authentication method to Git. Supported is a Username/Password combination (not recommended), an access token (recommended), and a base64 encoded SSH private key (recommended). When using the latter two, the username parameter is not required.

• A local Google Security Operations SOAR user. Used to import assets. This user must have permission to write the target module (For example, a user without access to the IDE would not be able to pull integrations).

Creating a new repository:

• All of the points mentioned in the Pushing/Pulling an existing repository above.

• A remote repository. It is recommended to have at least 1 file in the repository. Most Git services give an option to create a README file when creating the repository.

Integration Configuration

The integration doesn't need to be connected to an existing environment in Google Security Operations SOAR, so it's best to configure it as a Shared Instance.

Integration Properties

Configuring Repo URL and Branch

In this guide, we will demonstrate getting the right values in Bitbucket (noe that the process is the same in GitHub.)

1. Locate the repository in Bitbucket.

   

2. Click the clone button at the top right corner (Code in GitHub)

   • SSH Authentication - The repo url is git@bitbucket.org:siemplifyproserv/connectors.git

     

3. User/Pass or Token Authentication - The repo url is https://bitbucket.org/siemplifyproserv/connectors.git . (Username can be ignored)

   

4. Check the current branch (master in the image below)

   

Example Usage

Every Job in GitSync contains the following parameters:

Pulling Assets

In this example, we'll pull a connector with the correct mappings and visual families.

1. First, ensure the asset is located in the configured repository. Simply browse the repository directories and copy the asset identifier (Usually it's the directory name or the title of the README file).

   

   
   Example from a repo in Bitbucket, In the Connectors directory. Note that directories are the integration names, and inside them are the real identifiers for the connectors.

2. Find the suitable job in Google Security Operations SOAR IDE. In this example, we'll use the job Pull Connector.

   • Note: When pulling a connector, verify that the integration of the connector is installed as well.

3. Click on the testing tab and configure the parameters. Since we're using one repository and it's already configured in the integration instance, we'll leave the Repo URL and Branch parameters empty, and set the other parameters to the values we need.

4. Run the job.

5. See Debug Output for the log of the operation. If everything goes well, the log will indicate it.

6. Go to Google Security Operations SOAR -> Connectors and configure the connector.

Pushing Assets

In this example, we'll push a playbook and a block to the repository.

1. Identify the playbooks you want to push. Here we'll push a new block called Failed Login and an updated playbook called Malicious Activity.

   

   
   

2. Find the suitable job in Google Security Operations SOAR IDE. In this example, we'll use the job Push Playbook.

3. Click the testing tab and configure the parameters.

4. Since both are in the same folder (Default), You can also use Folder Allowlist instead.

5. Run the job.

6. See Debug Output for the log of the operation. If everything goes well, the log will indicate it.

7. Validate that the repository contains the latest versions of the playbooks.

Creating a new repository

To create a new repository, only one thing is important, and that is including a single file in the repo before configuring it with GitSync. It can be done quickly by including a README file in the root of the repository when creating the repository.
Bitbucket

GitHub

Known issues and limitations

• After the repository is set for the first time, It uses a predefined directory structure to ensure that it knows where each asset is located. Failing to follow the directory structure with a custom commit or changes to the repository, the integration will malfunction. You can find the schema of the repository directory structure at the end of this document.

• Be careful when using this integration with public repositories. Google Security Operations SOAR assets use parameters that hold Application IDs, Client ID, Usernames, and other sensitive information. GitSync can't tell if the parameter is sensitive or not, so every parameter that is not of type "Password" will be uploaded to the repository. Also, when pushing a Google Security Operations SOAR instance (Push Environment job) there's an option to Commit Passwords. This option tells GitSync to try and export all the password parameters from the integration configuration. Don't set this value to true if the repository is public, or all the credentials will be leaked online.

• When pulling a Google Security Operations SOAR instance (Pull Environment job), installing all the integrations might take more than 5 minutes, and the job will fail with a timeout. It is recommended to manually install all the commercial integration from the Google Security Operations Marketplace beforehand to avoid any issue, But it is also possible to re-run the job if it fails with a timeout.

• Commercial integrations and custom integrations are handled differently. Custom integration will be pushed as the entire zip export of the integration, for a simple import/export operation. Commercial integrations will be pushed with only the custom code. Once pulled, GitSync will install the latest version of the integration from the Google Security Operations Marketplace, and save the custom code in the official integration.

• When pulling mappings, they will not appear in the Settings -> Ontology -> Ontology Status table until the events have actually been ingested into Google Security Operations SOAR, because they are not indexed yet.

• The local repository is saved in /opt/siemplify/siemplify_server/GitSyncFiles/{RepoName}. Since the integration writes Git objects and not files, this folder doesn't represent the repository and will be overwritten with any changes made anytime a job is executed. It is advised to use another clone of the repository, and not the one created by GitSync.

Creating SSH Keys to use with GitSync

1. First, generate a key pair. When asked for a passphrase, hit enter: ssh-keygen -b 2048 -t rsa -f ./id_rsa

   Two files will be created, id_rsa (private key) and id_rsa.pub (public key). Keep the private key in a safe location.

2. Set the public key in the repository. For example, in Bitbucket, enter the settings of the repository and click on Access Keys. Click on Add Key and paste the contents of id_rsa.pub in the Key parameter.

3. Before the private key can be added to the integration configuration, it should be encoded to Base64.

   Use these commands to encode the file:

   • Linux:
     cat id_rsa | base64 -w 0
   • Windows:

     Open PowerShell where id_rsa is located and paste (This is one line):

     Write-Output [System.Text.Encoding]::ASCII.GetString([convert]::ToBase64String(([IO.File]::ReadAllBytes((Join-Path (pwd) 'id_rsa')))))
4. Copy the printed value to the integration property Password/Token/SSH Key and test the integration connectivity.

GitSync repository Directory Structure

The following is the expected directory structure in the remote repository.

* This is a tree command output for an example repo. Comments are in red.