Train and test models designed to detect money laundering

Create an instance

This section describes how to create an instance. The AML AI instance sits at the root of all other AML AI resources. Each instance requires a single associated customer-managed encryption key (CMEK) which is used to encrypt any data created by AML AI.

Create a key ring

To create a key ring, use the projects.locations.keyRings.create method.

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d "" \
     "https://cloudkms.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/keyRings?key_ring_id=my-key-ring"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -Uri "https://cloudkms.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/keyRings?key_ring_id=my-key-ring" | Select-Object -Expand Content

{
  "name": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring",
  "createTime": CREATE_TIME
}

gcloud kms keyrings create my-key-ring \
  --location LOCATION

gcloud kms keyrings create my-key-ring `
  --location LOCATION

gcloud kms keyrings create my-key-ring ^
  --location LOCATION

$

Create a key

To create a key, use the projects.locations.keyRings.cryptoKeys method.

{
  "purpose": "ENCRYPT_DECRYPT"
}

cat > request.json << 'EOF'
{
  "purpose": "ENCRYPT_DECRYPT"
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://cloudkms.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys?crypto_key_id=my-key"

@'
{
  "purpose": "ENCRYPT_DECRYPT"
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://cloudkms.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys?crypto_key_id=my-key" | Select-Object -Expand Content

{
  "name": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key",
  "primary": {
    "name": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key/cryptoKeyVersions/1",
    "state": "ENABLED",
    "createTime": CREATE_TIME,
    "protectionLevel": "SOFTWARE",
    "algorithm": "GOOGLE_SYMMETRIC_ENCRYPTION",
    "generateTime": GENERATE_TIME
  },
  "purpose": "ENCRYPT_DECRYPT",
  "createTime": CREATE_TIME,
  "versionTemplate": {
    "protectionLevel": "SOFTWARE",
    "algorithm": "GOOGLE_SYMMETRIC_ENCRYPTION"
  },
  "destroyScheduledDuration": "86400s"
}

gcloud kms keys create my-key \
  --keyring my-key-ring \
  --location LOCATION \
  --purpose "encryption"

gcloud kms keys create my-key `
  --keyring my-key-ring `
  --location LOCATION `
  --purpose "encryption"

gcloud kms keys create my-key ^
  --keyring my-key-ring ^
  --location LOCATION ^
  --purpose "encryption"

$

Create the instance using the API

To create an instance, use the projects.locations.instances.create method.

Request JSON body:

{
  "kmsKey": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key"
}

To send your request, choose one of these options:

cat > request.json << 'EOF'
{
  "kmsKey": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key"
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?instance_id=my-instance"

@'
{
  "kmsKey": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key"
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?instance_id=my-instance" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata",
    "createTime": CREATE_TIME,
    "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance",
    "verb": "create",
    "requestedCancellation": false,
    "apiVersion": "v1"
  },
  "done": false
}

If successful, the response body contains a long-running operation that contains an ID which can be used to retrieve the ongoing status of the asynchronous operation. Copy the returned OPERATION_ID to use in the next section.

Check for the result

Use the projects.locations.operations.get method to check if the instance has been created. If the response contains "done": false, repeat the command until the response contains "done": true.

Operations in this guide can take a few minutes to several hours to complete. You must wait until an operation completes before moving forward in this guide because the API uses the output of some methods as input to other methods.

Before using any of the request data, make the following replacements:

• OPERATION_ID: the identifier for the operation

To send your request, choose one of these options:

curl -X GET \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata",
    "createTime": CREATE_TIME,
    "endTime": END_TIME,
    "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance",
    "verb": "create",
    "requestedCancellation": false,
    "apiVersion": "v1"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.Instance",
    "name": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance",
    "createTime": CREATE_TIME,
    "updateTime": UPDATE_TIME,
    "kmsKey": "projects/KMS_PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key",
    "state": "ACTIVE"
  }
}

Grant access to the CMEK key

The API automatically creates a service account in your project. The service account needs access to the CMEK key so it can use the key to encrypt and decrypt the underlying data. Grant access to the key.

gcloud kms keys add-iam-policy-binding "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key" \
  --keyring "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring" \
  --location "LOCATION" \
  --member "serviceAccount:service-PROJECT_NUMBER@gcp-sa-financialservices.iam.gserviceaccount.com" \
  --role="roles/cloudkms.cryptoKeyEncrypterDecrypter" \
  --project="PROJECT_ID"

Create BigQuery datasets

This section describes how to create input and output BigQuery datasets, and then copy sample banking data into the input dataset.

Create an output dataset

Create a dataset to be used to send the AML pipeline outputs to.

bq mk \
  --location=LOCATION \
  --project_id=PROJECT_ID \
  my_bq_output_dataset

bq mk `
  --location=LOCATION `
  --project_id=PROJECT_ID `
  my_bq_output_dataset

Create an input dataset

Create a dataset to copy the sample banking tables into.

bq mk \
  --location=LOCATION \
  --project_id=PROJECT_ID \
  my_bq_input_dataset

bq mk `
  --location=LOCATION `
  --project_id=PROJECT_ID `
  my_bq_input_dataset

Copy the sample dataset

Sample banking data is provided as a BigQuery dataset in Google's shared dataset project. You must have access to the AML AI API for this dataset to be accessible. Key features of this dataset include the following:

• 100,000 parties
• A core time range from January 1, 2020 to January 1, 2023 and an additional 24 months of lookback data
• 300 negative and 20 positive risk cases per month
• Risk cases with the following attributes:
  • Half of the positive risk cases are for structuring activity which occurred in the two months preceding the AML_PROCESS_START event
  • The other half covers parties with the highest amount of received money in the two months preceding the AML_PROCESS_START event
  • Negative cases are randomly generated
  • A 0.1% chance for the risk case to be generated in the opposite state (for example, a random party that is positive, or a party that has structuring activity or the highest income and is reported negative)
• The AML schema is defined in the AML input data model.

1. Copy the sample banking data into the input dataset you created.

   Bash

   bq mk --transfer_config \
     --project_id=PROJECT_ID \
     --data_source=cross_region_copy \
     --target_dataset="my_bq_input_dataset" \
     --display_name="Copy the AML sample dataset." \
     --schedule=None \
     --params='{
       "source_project_id":"bigquery-public-data",
       "source_dataset_id":"aml_ai_input_dataset",
       "overwrite_destination_table":"true"
     }'
   

   PowerShell

   bq mk --transfer_config `
   --project_id=PROJECT_ID `
   --data_source=cross_region_copy `
   --target_dataset="my_bq_input_dataset" `
   --display_name="Copy the AML sample dataset." `
   --schedule=None `
   --params='{\"source_project_id\":\"bigquery-public-data\",\"source_dataset_id\":\"aml_ai_input_dataset\",\"overwrite_destination_table\":\"true\"}'
   

2. Monitor the data transfer job.

   Bash

   bq ls --transfer_config \
   --transfer_location=LOCATION \
   --project_id=PROJECT_ID \
   --filter="dataSourceIds:cross_region_copy"
   

   PowerShell

   bq ls --transfer_config `
   --transfer_location=LOCATION `
   --project_id=PROJECT_ID `
   --filter="dataSourceIds:cross_region_copy"
   

   After the transfer completes, a data transfer job with the display name Copy the AML sample dataset is created.

   You can also check the status of the transfer using the Google Cloud console.

Grant access to the BigQuery datasets

The API automatically creates a service account in your project. The service account needs access to the BigQuery input and output datasets.

1. Grant read access to the input dataset and its tables.

   Bash

   bq query --use_legacy_sql=false \
     'GRANT `roles/bigquery.dataViewer` ON SCHEMA `PROJECT_ID.my_bq_input_dataset` TO "serviceAccount:service-PROJECT_NUMBER@gcp-sa-financialservices.iam.gserviceaccount.com"'
   

   PowerShell

   bq query --use_legacy_sql=false "GRANT ``roles/bigquery.dataViewer`` ON SCHEMA ``PROJECT_ID.my_bq_input_dataset`` TO 'serviceAccount:service-PROJECT_NUMBER@gcp-sa-financialservices.iam.gserviceaccount.com'"
   

2. Grant write access to the output dataset.

   Bash

   bq query --use_legacy_sql=false \
     'GRANT `roles/bigquery.dataEditor` ON SCHEMA `PROJECT_ID.my_bq_output_dataset` TO "serviceAccount:service-PROJECT_NUMBER@gcp-sa-financialservices.iam.gserviceaccount.com"'
   

   PowerShell

   bq query --use_legacy_sql=false "GRANT ``roles/bigquery.dataEditor`` ON SCHEMA ``PROJECT_ID.my_bq_output_dataset`` TO 'serviceAccount:service-PROJECT_NUMBER@gcp-sa-financialservices.iam.gserviceaccount.com'"
   

Create an AML AI dataset

Create an AML AI dataset to specify the input BigQuery dataset tables and the time range to use.

To create a dataset, use the projects.locations.instances.datasets.create method.

Request JSON body:

{
  "tableSpecs": {
    "party": "bq://PROJECT_ID.my_bq_input_dataset.party",
    "account_party_link": "bq://PROJECT_ID.my_bq_input_dataset.account_party_link",
    "transaction": "bq://PROJECT_ID.my_bq_input_dataset.transaction",
    "risk_case_event": "bq://PROJECT_ID.my_bq_input_dataset.risk_case_event",
    "party_supplementary_data": "bq://PROJECT_ID.my_bq_input_dataset.party_supplementary_data"
  },
  "dateRange": {
    "startTime": "2020-01-01T00:00:0.00Z",
    "endTime": "2023-01-01T00:00:0.00Z"
  },
  "timeZone": {
    "id": "UTC"
  }
}

To send your request, choose one of these options:

cat > request.json << 'EOF'
{
  "tableSpecs": {
    "party": "bq://PROJECT_ID.my_bq_input_dataset.party",
    "account_party_link": "bq://PROJECT_ID.my_bq_input_dataset.account_party_link",
    "transaction": "bq://PROJECT_ID.my_bq_input_dataset.transaction",
    "risk_case_event": "bq://PROJECT_ID.my_bq_input_dataset.risk_case_event",
    "party_supplementary_data": "bq://PROJECT_ID.my_bq_input_dataset.party_supplementary_data"
  },
  "dateRange": {
    "startTime": "2020-01-01T00:00:0.00Z",
    "endTime": "2023-01-01T00:00:0.00Z"
  },
  "timeZone": {
    "id": "UTC"
  }
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets?dataset_id=my-dataset"

@'
{
  "tableSpecs": {
    "party": "bq://PROJECT_ID.my_bq_input_dataset.party",
    "account_party_link": "bq://PROJECT_ID.my_bq_input_dataset.account_party_link",
    "transaction": "bq://PROJECT_ID.my_bq_input_dataset.transaction",
    "risk_case_event": "bq://PROJECT_ID.my_bq_input_dataset.risk_case_event",
    "party_supplementary_data": "bq://PROJECT_ID.my_bq_input_dataset.party_supplementary_data"
  },
  "dateRange": {
    "startTime": "2020-01-01T00:00:0.00Z",
    "endTime": "2023-01-01T00:00:0.00Z"
  },
  "timeZone": {
    "id": "UTC"
  }
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets?dataset_id=my-dataset" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata",
    "createTime": CREATE_TIME,
    "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "verb": "create",
    "requestedCancellation": false,
    "apiVersion": "v1"
  },
  "done": false
}

You can check for the result of the operation using the new operation ID. (You can do this for the remaining API requests used in this guide.)

Create an engine config

Create an AML AI engine config to automatically tune hyperparameters based on a given engine version and the data provided. Engine versions are released periodically and correspond to different model logic (for example, targeting a retail line of business versus a commercial one).

To create an engine config, use the projects.locations.instances.engineConfigs.create method.

This stage involves hyperparameter tuning which can take some time to process. Provided your data does not change substantially, this step can be used to create and test many models.

Request JSON body:

{
  "engineVersion": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineVersions/aml-commercial.default.v004.000.202312-000",
  "tuning": {
    "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2021-07-01T00:00:00Z"
  },
  "performanceTarget": {
    "partyInvestigationsPerPeriodHint": "30"
  }
}

To send your request, choose one of these options:

cat > request.json << 'EOF'
{
  "engineVersion": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineVersions/aml-commercial.default.v004.000.202312-000",
  "tuning": {
    "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2021-07-01T00:00:00Z"
  },
  "performanceTarget": {
    "partyInvestigationsPerPeriodHint": "30"
  }
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs?engine_config_id=my-engine-config"

@'
{
  "engineVersion": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineVersions/aml-commercial.default.v004.000.202312-000",
  "tuning": {
    "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2021-07-01T00:00:00Z"
  },
  "performanceTarget": {
    "partyInvestigationsPerPeriodHint": "30"
  }
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs?engine_config_id=my-engine-config" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata",
    "createTime": CREATE_TIME,
    "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config",
    "verb": "create",
    "requestedCancellation": false,
    "apiVersion": "v1"
  },
  "done": false
}

Create a model

In this step, you train an AML AI model by using 12 months of data leading up to 2021-07-01.

To create a model, use the projects.locations.instances.models.create method.

Request JSON body:

{
    "engineConfig": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config",
    "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2021-07-01T00:00:00Z"
}

To send your request, choose one of these options:

cat > request.json << 'EOF'
{
    "engineConfig": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config",
    "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2021-07-01T00:00:00Z"
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models?model_id=my-model"

@'
{
    "engineConfig": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config",
    "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2021-07-01T00:00:00Z"
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models?model_id=my-model" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata",
    "createTime": CREATE_TIME,
    "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model",
    "verb": "create",
    "requestedCancellation": false,
    "apiVersion": "v1"
  },
  "done": false
}

Create a backtest result

Backtest prediction uses the trained model on existing historical data. Create a backtest result on the 12 months of data leading up to January 2023, which were not used in training. These months are used to determine how many cases we might need to work had we used the model trained to July 2021 in production during January to December 2022.

To create a backtest result, use the projects.locations.instances.backtestResults.create method.

Request JSON body:

{
    "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model",
    "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2023-01-01T00:00:00Z",
    "backtestPeriods": 12,
    "performanceTarget": {
      "partyInvestigationsPerPeriodHint": "150"
    }
}

To send your request, choose one of these options:

cat > request.json << 'EOF'
{
    "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model",
    "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2023-01-01T00:00:00Z",
    "backtestPeriods": 12,
    "performanceTarget": {
      "partyInvestigationsPerPeriodHint": "150"
    }
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults?backtest_result_id=my-backtest-results"

@'
{
    "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model",
    "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2023-01-01T00:00:00Z",
    "backtestPeriods": 12,
    "performanceTarget": {
      "partyInvestigationsPerPeriodHint": "150"
    }
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults?backtest_result_id=my-backtest-results" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata",
    "createTime": CREATE_TIME,
    "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results",
    "verb": "create",
    "requestedCancellation": false,
    "apiVersion": "v1"
  },
  "done": false
}

Export backtest results metadata

Once a backtest has been run, you need to export its results to BigQuery to view them. To export metadata from the backtest result, use the projects.locations.instances.backtestResults.exportMetadata method.

Request JSON body:

{
  "structuredMetadataDestination": {
    "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_backtest_results_metadata",
    "writeDisposition": "WRITE_TRUNCATE"
  }
}

To send your request, choose one of these options:

cat > request.json << 'EOF'
{
  "structuredMetadataDestination": {
    "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_backtest_results_metadata",
    "writeDisposition": "WRITE_TRUNCATE"
  }
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results:exportMetadata"

@'
{
  "structuredMetadataDestination": {
    "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_backtest_results_metadata",
    "writeDisposition": "WRITE_TRUNCATE"
  }
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results:exportMetadata" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata",
    "createTime": CREATE_TIME,
    "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results",
    "verb": "exportMetadata",
    "requestedCancellation": false,
    "apiVersion": "v1"
  },
  "done": false
}

Once the operation has completed, do the following:

1. Open BigQuery in the Google Cloud console.

   Go to the Google Cloud console

2. Find and expand the output dataset in the Explorer pane.

3. Select the table and click Preview.

4. Find the row with the name ObservedRecallValues.

   

5. Assume that your capacity for investigations is 120 per month. Find the recall value object with "partyInvestigationsPerPeriod": "120"`. For the following sample values, if you limit investigations to parties with risk scores greater than 0.53, then you can expect to investigate 120 new parties each month. Over the backtesting period, the year 2022, you would identify 86% of cases that the previous system identified (and possibly others, which were not identified by your current processes).

   {
     "recallValues": [
       ...
       {
         "partyInvestigationsPerPeriod": "105",
         "recallValue": 0.8142077,
         "scoreThreshold": 0.6071321
       },
       {
         "partyInvestigationsPerPeriod": "120",
         "recallValue": 0.863388,
         "scoreThreshold": 0.5339603
       },
       {
         "partyInvestigationsPerPeriod": "135",
         "recallValue": 0.89071035,
         "scoreThreshold": 0.4739899
       },
       ...
     ]
   }
   

See more about the other fields in the backtest results.

By changing the partyInvestigationsPerPeriodHint field, you can modify the number of investigations a backtest produces. To obtain scores to investigate, register parties and produce predictions against them.

Import registered parties

Before creating prediction results, you need to import registered parties (that is, customers in the dataset).

To import registered parties, use the projects.locations.instances.importRegisteredParties method.

Request JSON body:

{
  "partyTables": [
     "bq://PROJECT_ID.my_bq_input_dataset.party_registration"
  ],
  "mode": "REPLACE",
  "lineOfBusiness": "COMMERCIAL"
}

To send your request, choose one of these options:

cat > request.json << 'EOF'
{
  "partyTables": [
     "bq://PROJECT_ID.my_bq_input_dataset.party_registration"
  ],
  "mode": "REPLACE",
  "lineOfBusiness": "COMMERCIAL"
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance:importRegisteredParties"

@'
{
  "partyTables": [
     "bq://PROJECT_ID.my_bq_input_dataset.party_registration"
  ],
  "mode": "REPLACE",
  "lineOfBusiness": "COMMERCIAL"
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance:importRegisteredParties" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata",
    "createTime": CREATE_TIME,
    "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance",
    "verb": "importRegisteredParties",
    "requestedCancellation": false,
    "apiVersion": "v1"
  },
  "done": false
}

When the operation completes, you should see that 100,000 parties were registered.

Create a prediction result

Create a prediction result on the last 12 months in the dataset; these months were not used during training. Creating prediction results creates scores for each party in each month across all the prediction periods.

To create a prediction result, use the projects.locations.instances.predictionResults.create method.

Request JSON body:

{
    "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model",
    "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2023-01-01T00:00:00Z",
    "predictionPeriods": "12",
    "outputs": {
      "predictionDestination": {
        "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results",
        "writeDisposition": "WRITE_TRUNCATE"
      },
      "explainabilityDestination": {
        "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results_explainability",
        "writeDisposition": "WRITE_TRUNCATE"
      }
    }
}

To send your request, choose one of these options:

cat > request.json << 'EOF'
{
    "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model",
    "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2023-01-01T00:00:00Z",
    "predictionPeriods": "12",
    "outputs": {
      "predictionDestination": {
        "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results",
        "writeDisposition": "WRITE_TRUNCATE"
      },
      "explainabilityDestination": {
        "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results_explainability",
        "writeDisposition": "WRITE_TRUNCATE"
      }
    }
}
EOF

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/predictionResults?prediction_result_id=my-prediction-results"

@'
{
    "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model",
    "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset",
    "endTime": "2023-01-01T00:00:00Z",
    "predictionPeriods": "12",
    "outputs": {
      "predictionDestination": {
        "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results",
        "writeDisposition": "WRITE_TRUNCATE"
      },
      "explainabilityDestination": {
        "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results_explainability",
        "writeDisposition": "WRITE_TRUNCATE"
      }
    }
}
'@  | Out-File -FilePath request.json -Encoding utf8

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/predictionResults?prediction_result_id=my-prediction-results" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata",
    "createTime": CREATE_TIME,
    "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/predictionResults/my-prediction-results",
    "verb": "create",
    "requestedCancellation": false,
    "apiVersion": "v1"
  },
  "done": false
}

Analyze a single structuring case in the Google Cloud console

1. Open BigQuery in the Google Cloud console. Make sure SQL workspace is selected.

   Go to the Google Cloud console

2. The BigQuery page has three main sections:

   • The BigQuery navigation menu
   • The Explorer pane
   • The details pane

   In the details pane, click Compose a New Query to open the query editor.

   

3. Copy the following SQL statement into the editor and click Run.

   SELECT *
   FROM `PROJECT_ID.my_bq_input_dataset.transaction`
   WHERE account_id = '1E60OAUNKP84WDKB' AND DATE_TRUNC(book_time, MONTH) = "2022-08-01"
   ORDER by book_time
   

   This statement checks account ID 1E60OAUNKP84WDKB in August 2022. This account is linked to party ID EGS4NJD38JZ8NTL8. You can find the party ID for a given account ID by using the AccountPartyLink table.

   The transaction data shows frequent round transactions targeted at a single account shortly after large cash deposits, which looks suspicious. These transactions could be indicative of smurfing (that is, breaking up a large transaction of money into smaller transaction amounts) or structuring.

   

4. Copy the following SQL statement into the editor and click Run.

   SELECT *
   FROM `PROJECT_ID.my_bq_input_dataset.risk_case_event`
   WHERE party_id = 'EGS4NJD38JZ8NTL8'
   

   This statement shows that there was a risk case leading to the exit of this party. The risk case started two months after the suspicious activity.

   

5. Copy the following SQL statement into the editor and click Run.

   SELECT *
   FROM `PROJECT_ID.my_bq_output_dataset.my_prediction_results`
   WHERE party_id = 'EGS4NJD38JZ8NTL8'
   ORDER BY risk_period_end_time
   

   By checking the prediction results, you can see that the party's risk score jumps from nearly zero (note the exponent value) to high values in the months following the suspicious activity. Your results may vary from the results shown.

   

   The risk score is not a probability. A risk score should always be evaluated relative to other risk scores. For example, a seemly small value can be considered positive in cases where the other risk scores are lower.

6. Copy the following SQL statement into the editor and click Run.

   SELECT *
   FROM `PROJECT_ID.my_bq_output_dataset.my_prediction_results_explainability`
   WHERE party_id = 'EGS4NJD38JZ8NTL8'
   AND risk_period_end_time = '2022-10-01'
   

   By checking the explainability results, you can see that the correct feature families score the highest values.