Configure and manage Storage Intelligence

This page describes how to configure, view, update, and disable Storage Intelligence for a resource.

Configure Storage Intelligence

This section describes how to configure Storage Intelligence for a resource.

To get the required permission for configuring Storage Intelligence, ask your administrator to grant you any of the following IAM roles on the Google Cloud Resource Hierarchy (projects, folders, and organizations):

  • Owner (roles/owner)
  • Storage Admin (roles/storage.admin)

These roles contain the storage.intelligenceConfigs.update permission, which is required to configure Storage Intelligence.
You can also get this permission with custom roles. You might be able to get this permission with other predefined roles as well. To see which roles are associated with which permissions, refer to IAM roles for Cloud Storage.

To grant, change, or revoke access to projects, folders, and organizations, see Manage access to projects, folders, and organizations.

  1. In the Google Cloud console, go to the Cloud Storage Settings page.

    Go to Settings

  2. Click Configure Storage Intelligence.

  3. In the Configure Storage Intelligence dialog, select the resource that you want to configure Storage Intelligence for.

  4. Click Continue.

  5. Optional: To include or exclude specific buckets in your Storage Intelligence configuration, do the following:

    1. To select buckets from a specific locations, use the filter type menu. To include or exclude buckets, select the location from the location menu. For more information, see Filter by location.

    2. To select Cloud Storage buckets by name, use the filter type menu. To include or exclude buckets, specify the bucket name's regular expression and click Add regular expression. For more information, see Filter by bucket name.

  6. Click Enable this configuration.

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. To configure Storage Intelligence, use the gcloud alpha storage intelligence-configs enable command with one of the following flags:

    • --project: Configure Storage Intelligence for a project. The value is a project ID. For example, my-project.

    • --sub-folder: Configure Storage Intelligence for a folder. The value is a folder ID. For example, 1234567890.

    • --organization: Configure Storage Intelligence for an organization. The value is an organization ID. For example, 9876546738.

      For example, the following command configures Storage Intelligence for a project called my-project:

      gcloud alpha storage intelligence-configs enable --project=PROJECT_ID

      Where PROJECT_ID is a project ID.

    To include or exclude specific buckets in your Storage Intelligence configuration, use any one of the following flags:

    • --exclude-bucket-regexes: List of bucket ID regexes to be excluded from the Storage Intelligence configuration.
    • --include-bucket-regexes: List of bucket ID regexes to be included in the Storage Intelligence configuration.

    For example, to exclude all buckets that contain colddata in their names, use the following command:

    gcloud alpha storage intelligence-configs enable --organization=54343 --exclude-bucket-regexes=colddata.*

    For example, to exclude buckets named colddata, use the following command:

    gcloud alpha storage intelligence-configs enable --organization=54343 --exclude-bucket-regexes=colddata

    For example, to exclude buckets named colour or color, use the following command:

    gcloud alpha storage intelligence-configs enable --organization=my-org --exclude-bucket-regexes=colou?r

    To include or exclude buckets based on their location in your Storage Intelligence configuration, use any one of the following flags:

    • --exclude-locations: List of locations to be excluded from the Storage Intelligence configuration.
    • --include-locations: List of locations to be included in the Storage Intelligence configuration.

    For example:

    gcloud alpha storage intelligence-configs enable --sub-folder=123456 --include-locations=us-east1,us-west1
JSON API
  1. Have gcloud CLI installed and initialized, which lets you generate an access token for the Authorization header.

  2. Create a JSON file with the following details:

    1. name: The name of the Storage Intelligence configuration. Use any one of the following formats:
      • Project: projects/PROJECT_ID/locations/global/intelligenceConfig
      • Folder: folders/FOLDER_ID/locations/global/intelligenceConfig
      • Organization: organizations/ORGANIZATION_ID/locations/global/intelligenceConfig

      Where:

      • PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project
      • FOLDER_ID is the Google Cloud folder ID. Folder IDs are numeric, for example, 123456789012
      • ORGANIZATION_ID is the Google Cloud organization ID. Organization IDs are numeric, for example, 123456789012
    2. Configure the editionConfig field. For more information, see Edition configuration.
    3. Optional: Configure the filter object. You can include or exclude buckets based on Cloud Storage locations and bucket ID regular expressions.
        {
          "name":"projects/PROJECT_ID/locations/global/intelligenceConfig",
          "editionConfig":"EDITION_CONFIGURATION",
          "filter":{
            "includedCloudStorageBuckets":{
                "bucketIdRegexes":[
                  "INCLUDED_BUCKET_ID_REGEX",
                    ...]
            },
            "excludedCloudStorageBuckets":{
                "bucketIdRegexes":[
                  "EXCLUDED_BUCKET_ID_REGEX",
                    ...]
            },
            "includedCloudStorageLocations":{
                "locations":[
                  "INCLUDED_BUCKET_LOCATION",
                    ...]
            },
            "excludedCloudStorageLocations":{
                "locations":[
                  "EXCLUDED_BUCKET_LOCATION",
                    ...]
            }
          }
        }
      

    Where:

    • PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project.
    • EDITION_CONFIGURATION is the Storage Intelligence edition configuration.
    • Optional:INCLUDED_BUCKET_ID_REGEX is a regular expression that specifies which bucket IDs to include. You can specify one or multiple regular expressions as a list.
    • Optional: EXCLUDED_BUCKET_ID_REGEX is a regular expression that specifies which bucket IDs to exclude. You can specify one or multiple regular expressions as a list.
    • Optional: INCLUDED_BUCKET_LOCATION is the location where you want to include buckets in the Storage Intelligence configuration. You can specify one or multiple locations as a list.
    • Optional: EXCLUDED_BUCKET_LOCATION is the location where you want to exclude buckets from the Storage Intelligence configuration. You can specify one or multiple locations as a list.
  3. The following sections describe how to configure Storage Intelligence for a project, folder, or an organization. Configure Storage Intelligence using any one of the following methods.

    Configure Storage Intelligence for a project

    Use cURL to call the JSON API with a request to configure Storage Intelligence for a project.

      curl -X POST \
        --data-binary @JSON_FILE_NAME \
        -H "Authorization: Bearer $(gcloud auth print-access-token)" \
        -H "Content-Type: application/json" \
        "https://storage.googleapis.com/v2/projects/PROJECT_ID/locations/global/intelligenceConfig?updateMask=UPDATE_MASK"
    

    Where:

    • PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project
    • UPDATE_MASK is the comma-separated list of field names that this request updates. The fields use the FieldMask format and are part of the IntelligenceConfig resource.

    Configure Storage Intelligence for a folder

    Use cURL to call the JSON API with a request to configure Storage Intelligence for a folder.

      curl -X POST \
        --data-binary @JSON_FILE_NAME \
        -H "Authorization: Bearer $(gcloud auth print-access-token)" \
        -H "Content-Type: application/json" \
        "https://storage.googleapis.com/v2/folders/FOLDER_ID/locations/global/intelligenceConfig?updateMask=UPDATE_MASK"
    

    Where:

    • FOLDER_ID is the Google Cloud folder ID. Folder IDs are numeric, for example, 123456789012
    • UPDATE_MASK is the comma-separated list of field names that this request updates. The fields use the FieldMask format and are part of the IntelligenceConfig resource.

    Configure Storage Intelligence for an organization

    Use cURL to call the JSON API with a request to configure Storage Intelligence for an organization.

    curl -X POST \
      --data-binary @JSON_FILE_NAME \
      -H "Authorization: Bearer $(gcloud auth print-access-token)" \
      -H "Content-Type: application/json" \
      "https://storage.googleapis.com/v2/organizations/ORGANIZATION_ID/locations/global/intelligenceConfig?updateMask=UPDATE_MASK"
    

    Where:

    • ORGANIZATION_ID is the Google Cloud organization ID. Organization IDs are numeric, for example, 79293024087
    • UPDATE_MASK is the comma-separated list of field names that this request updates. The fields use the FieldMask format and are part of the IntelligenceConfig resource.

View Storage Intelligence configuration

This section describes how to view the Storage Intelligence configuration for a resource.

To get the required permission for viewing Storage Intelligence configuration, ask your administrator to grant you any of the following IAM roles on the Google Cloud Resource Hierarchy (projects, folders, and organizations):

  • Owner (roles/owner)
  • Storage Admin (roles/storage.admin)

These roles contain the storage.intelligenceConfigs.get permission, which is required to view Storage Intelligence configuration.
You can also get this permission with custom roles. You might be able to get this permission with other predefined roles as well. To see which roles are associated with which permissions, refer to IAM roles for Cloud Storage.

To grant, change, or revoke access to projects, folders, and organizations, see Manage access to projects, folders, and organizations.

  1. In the Google Cloud console, go to the Cloud Storage Settings page.

    Go to Settings

  2. Select the project picker at the top of the page.

  3. In the Select a resource dialog, select the resource for which you want to view the Storage Intelligence configuration.

If a resource has Storage Intelligence configured, the Status field displays Enabled. For resources that don't have Storage Intelligence configured, the Configure Storage Intelligence button is displayed.

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. To view the Storage Intelligence configuration for a resource, use the gcloud alpha storage intelligence-configs describe command with one of the following flags:

    • --project: View the Storage Intelligence configuration for a project. The value is a project ID. For example, my-project.

    • --sub-folder: View the Storage Intelligence configuration for a folder. The value is a folder ID. For example, 1234567890.

    • --organization: View the Storage Intelligence configuration for an organization. The value is an organization ID. For example, 9876546738.

      For example, the following command views the Storage Intelligence configuration for a project called my-project:

      gcloud alpha storage intelligence-configs describe --project=PROJECT_ID

      Where PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project.

JSON API
  1. Have gcloud CLI installed and initialized, which lets you generate an access token for the Authorization header.

  2. The methods for viewing the Storage Intelligence configuration for a resource are detailed in the following sections.

    View the Storage Intelligence configuration for a project

    Use cURL to call the JSON API with a request to view the Storage Intelligence configuration for a project.

      curl -X GET \
        -H "Authorization: Bearer $(gcloud auth print-access-token)" \
        "https://storage.googleapis.com/v2/projects/PROJECT_ID/locations/global/intelligenceConfig"
    

    Where:

    PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project.

    View the Storage Intelligence configuration for a folder

    Use cURL to call the JSON API with a request to view the Storage Intelligence configuration for a folder.

      curl -X GET \
        -H "Authorization: Bearer $(gcloud auth print-access-token)" \
        "https://storage.googleapis.com/v2/folders/FOLDER_ID/locations/global/intelligenceConfig"
    

    Where:

    FOLDER_ID is the Google Cloud folder ID. Folder IDs are numeric, for example, 123456789012.

    View the Storage Intelligence configuration for an organization

    Use cURL to call the JSON API with a request to view the Storage Intelligence configuration for an organization.

      curl -X GET \
        -H "Authorization: Bearer $(gcloud auth print-access-token)" \
        "https://storage.googleapis.com/v2/organizations/ORGANIZATION_ID/locations/global/intelligenceConfig"
    

    Where:

    ORGANIZATION_ID is the Google Cloud organization ID. Organization IDs are numeric, for example, 79293024087.

Update Storage Intelligence configuration

This section describes how to update the Storage Intelligence configuration for a resource.

To get the required permission for updating Storage Intelligence configuration, ask your administrator to grant you any of the following IAM roles on the Google Cloud Resource Hierarchy (projects, folders, and organizations):

  • Owner (roles/owner)
  • Storage Admin (roles/storage.admin)

These roles contain the storage.intelligenceConfigs.update permission, which is required to update Storage Intelligence configuration.
You can also get this permission with custom roles. You might be able to get this permission with other predefined roles as well. To see which roles are associated with which permissions, refer to IAM roles for Cloud Storage.

To grant, change, or revoke access to projects, folders, and organizations, see Manage access to projects, folders, and organizations.

  1. In the Google Cloud console, go to the Cloud Storage Settings page.

    Go to Settings

  2. Select the project picker at the top of the page.

  3. In the Select a resource dialog, select the resource for which you want to update the Storage Intelligence configuration.

  4. In the Overview section, modify any of the editable configuration options to meet your needs. For example, you can disable Storage Intelligence for a resource or change the edition configuration to Inherit from parent or change the bucket filters.

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. To update the Storage Intelligence configuration for a resource, use the gcloud alpha storage intelligence-configs update command with one of the following flags:

    • --project: Update the Storage Intelligence configuration for a project. The value is a project ID. For example, my-project.

    • --sub-folder: Update the Storage Intelligence configuration for a folder. The value is a folder ID. For example, 1234567890.

    • --organization: Update the Storage Intelligence configuration for an organization. The value is an organization ID. For example, 9876546738.

      For example, the following command updates the Storage Intelligence configuration for a project called my-project:

      gcloud alpha storage intelligence-configs update --project=PROJECT_ID

      Where PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project.

JSON API
  1. Have gcloud CLI installed and initialized, which lets you generate an access token for the Authorization header.

  2. Create a JSON file with the following details:

    1. name: The name of the Storage Intelligence configuration. Use any one of the following formats:
      • Project: projects/PROJECT_ID/locations/global/intelligenceConfig
      • Folder: folders/FOLDER_ID/locations/global/intelligenceConfig
      • Organization: organizations/ORGANIZATION_ID/locations/global/intelligenceConfig

      Where:

      • PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project
      • FOLDER_ID is the Google Cloud folder ID. Folder IDs are numeric, for example, 123456789012
      • ORGANIZATION_ID is the Google Cloud organization ID. Organization IDs are numeric, for example, 123456789012
    2. Configure the editionConfig field. For more information, see Edition configuration.
    3. Optional: Configure the filter object. You can include or exclude buckets based on Cloud Storage locations and bucket ID regular expressions.
        {
          "name":"projects/PROJECT_ID/locations/global/intelligenceConfig",
          "editionConfig":"EDITION_CONFIGURATION",
          "filter":{
            "includedCloudStorageBuckets":{
                "bucketIdRegexes":[
                  "INCLUDED_BUCKET_ID_REGEX",
                    ...]
            },
            "excludedCloudStorageBuckets":{
                "bucketIdRegexes":[
                  "EXCLUDED_BUCKET_ID_REGEX",
                    ...]
            },
            "includedCloudStorageLocations":{
                "locations":[
                  "INCLUDED_BUCKET_LOCATION",
                    ...]
            },
            "excludedCloudStorageLocations":{
                "locations":[
                  "EXCLUDED_BUCKET_LOCATION",
                    ...]
            }
          }
        }
      

    Where:

    • PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project.
    • EDITION_CONFIGURATION is the Storage Intelligence edition configuration.
    • Optional:INCLUDED_BUCKET_ID_REGEX is a regular expression that specifies which bucket IDs to include. You can specify one or multiple regular expressions as a list.
    • Optional: EXCLUDED_BUCKET_ID_REGEX is a regular expression that specifies which bucket IDs to exclude. You can specify one or multiple regular expressions as a list.
    • Optional: INCLUDED_BUCKET_LOCATION is the location where you want to include buckets in the Storage Intelligence configuration. You can specify one or multiple locations as a list.
    • Optional: EXCLUDED_BUCKET_LOCATION is the location where you want to exclude buckets from the Storage Intelligence configuration. You can specify one or multiple locations as a list.
  3. The following sections describe how to update the Storage Intelligence configuration for a project, folder, or an organization. Update the Storage Intelligence configuration using one of these methods.

    Update the Storage Intelligence configuration for a project

    Use cURL to call the JSON API with a request to update the Storage Intelligence configuration for a project.

     curl -X PATCH \
       --data-binary @JSON_FILE_NAME \
       -H "Authorization: Bearer $(gcloud auth print-access-token)" \
       -H "Content-Type: application/json" \
       "https://storage.googleapis.com/v2/projects/PROJECT_ID/locations/global/intelligenceConfig?updateMask=UPDATE_MASK"
    

    Update the Storage Intelligence configuration for a folder

    Use cURL to call the JSON API with a request to update the Storage Intelligence configuration for a folder.

    Where:

    • PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project
    • UPDATE_MASK is the comma-separated list of field names that this request updates. The fields use the FieldMask format and are part of the IntelligenceConfig resource.

     curl -X PATCH \
       --data-binary @JSON_FILE_NAME \
       -H "Authorization: Bearer $(gcloud auth print-access-token)" \
       -H "Content-Type: application/json" \
       "https://storage.googleapis.com/v2/folders/FOLDER_ID/locations/global/intelligenceConfig?updateMask=UPDATE_MASK"
    

    Where:

    • FOLDER_ID is the Google Cloud folder ID. Folder IDs are numeric, for example, 123456789012
    • UPDATE_MASK is the comma-separated list of field names that this request updates. The fields use the FieldMask format and are part of the IntelligenceConfig resource.

    Update the Storage Intelligence configuration for an organization

    Use cURL to call the JSON API with a request to update the Storage Intelligence configuration for an organization.

     curl -X PATCH \
       --data-binary @JSON_FILE_NAME \
       -H "Authorization: Bearer $(gcloud auth print-access-token)" \
       -H "Content-Type: application/json" \
       "https://storage.googleapis.com/v2/organizations/ORGANIZATION_ID/locations/global/intelligenceConfig?updateMask=UPDATE_MASK"
    

    Where:

    • ORGANIZATION_ID is the Google Cloud organization ID. Organization IDs are numeric, for example, 79293024087
    • UPDATE_MASK is the comma-separated list of field names that this request updates. The fields use the FieldMask format and are part of the IntelligenceConfig resource.

Disable Storage Intelligence

This section describes how to disable Storage Intelligence.

To get the required permission for disabling Storage Intelligence, ask your administrator to grant you any of the following IAM roles on the Google Cloud Resource Hierarchy (projects, folders, and organizations):

  • Owner (roles/owner)
  • Storage Admin (roles/storage.admin)

These roles contain the storage.intelligenceConfigs.update permission, which is required to disable Storage Intelligence.
You can also get this permission with custom roles. You might be able to get this permission with other predefined roles as well. To see which roles are associated with which permissions, refer to IAM roles for Cloud Storage.

To grant, change, or revoke access to projects, folders, and organizations, see Manage access to projects, folders, and organizations.

  1. In the Google Cloud console, go to the Cloud Storage Settings page.

    Go to Settings

  2. Select the project picker at the top of the page.

  3. In the Select a resource dialog, select the resource for which you want to disable Storage Intelligence.

  4. In the Overview section, click Enabled next to the Status field.

  5. In the dialog that appears, enter Disable and then click Disable.

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. To disable the Storage Intelligence configuration for a resource, use the gcloud alpha storage intelligence-configs disable command with one of the following flags:

    • --project: Disable the Storage Intelligence configuration for a project. The value is a project ID. For example, my-project.

    • --sub-folder: Disable the Storage Intelligence configuration for a folder. The value is a folder ID. For example, 1234567890.

    • --organization: Disable the Storage Intelligence configuration for an organization. The value is an organization ID. For example, 9876546738.

      For example, the following command disables the Storage Intelligence configuration for a project called my-project:

      gcloud alpha storage intelligence-configs disable --project=PROJECT_ID

      Where PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project.

JSON API
  1. Have gcloud CLI installed and initialized, which lets you generate an access token for the Authorization header.

  2. The methods to disable Storage Intelligence for a resource are detailed in the following sections.

    Disable Storage Intelligence for a project

    Use cURL to call the JSON API with a request to update Storage Intelligence for a project.

    curl -X PATCH \
     -H "Content-Type: application/json" \
     -d '{
          "edition_config": "DISABLED"
         }' \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
      "https://storage.googleapis.com/v2/projects/PROJECT_ID/locations/global/intelligenceConfig?updateMask=edition_config"

    Where:
    PROJECT_ID is the Google Cloud project ID. Project IDs are alphanumeric, for example, my-project.

    Disable Storage Intelligence for a folder

    Use cURL to call the JSON API with a request to update Storage Intelligence for a folder.

    curl -X PATCH \
      -H "Content-Type: application/json" \
      -d '{
        "edition_config": "DISABLED"
      }' \
      -H "Authorization: Bearer $(gcloud auth print-access-token)" \
      "https://storage.googleapis.com/v2/folders/FOLDER_ID/locations/global/intelligenceConfig?updateMask=edition_config"
    

    Where:
    FOLDER_ID is the Google Cloud folder ID. Folder IDs are numeric, for example, 123456789012.

    Disable Storage Intelligence for an organization

    Use cURL to call the JSON API with a request to disable Storage Intelligence for an organization.

    curl -X PATCH \
      -H "Content-Type: application/json" \
      -d '{
        "edition_config": "DISABLED"
      }' \
      -H "Authorization: Bearer $(gcloud auth print-access-token)" \
      "https://storage.googleapis.com/v2/organizations/ORGANIZATION_ID/locations/global/intelligenceConfig?updateMask=edition_config"

    Where:
    ORGANIZATION_ID is the Google Cloud organization ID. Organization IDs are numeric, for example, 79293024087.

What's next