Change environment networking type (Private or Public IP)

Cloud Composer 3 | Cloud Composer 2 | Cloud Composer 1

This page explains the difference between Private IP and Public IP environment networking types in Cloud Composer 3 and provides instructions for switching the networking type of your environment.

If you want to disable or enable internet access only when installing PyPI packages, see Configure internet access when installing PyPI packages.

If you want to enable access to your VPC network from your environment, see Connect an environment to a VPC network.

About environment networking types

Cloud Composer 3 uses two environment networking types:

  • Public IP networking:

    • Airflow components of the environment can access the internet. This is the default networking type.

    • Airflow components in Public IP environments establish outbound connections from automatically allocated public IP addresses and ports. If you want your Public IP environments to use predetermined IP addresses and ports, you can do so through connecting a VPC network to your environment and switching it to Private IP. In this case, Cloud Composer routes all traffic except traffic to Google services through this network.

  • Private IP networking:

    • Airflow components of the environment don't have access to the internet.

    • Private IP environments configure Private Google Access through the private.googleapis.com range, which enables access to Google APIs, services, and domains supported by this range.

      For more information and the list of services and domains available through private.googleapis.com, see Network configuration in the Virtual Private Cloud documentation.

    • Private IP environments with VPC Service Controls configure Private Google Access through the restricted.googleapis.com range, which enables access to Google APIs, services, and domains supported by this range.

      For more information and the list of services and domains available through resrticted.googleapis.com, see Network configuration in the Virtual Private Cloud documentation.

In addition to two networking types, you can enable or disable access to a custom VPC network for any type of environment. Depending on how you configure your VPC network, a Private IP environment can gain access the internet through you VPC network.

Cloud Composer 2 networking compared to Cloud Composer 3

In Cloud Composer 3, Private IP environments require no configuration.

The following Cloud Composer 2 networking features are no longer relevant in Cloud Composer 3:

  • Configuring Private IP networking. You don't need to specify IP ranges, networks, or configure connectivity and firewall rules.

  • Configuring Private Service Connect. You don't need to set ranges for Private Service Connect in Cloud Composer 3.

  • Using privately used public IP ranges. This feature provided an option to extend the available IP ranges, which are not required in Cloud Composer 3.

  • Using the IP Masquerade agent. You don't need to configure cluster connectivity in Cloud Composer 3.

  • Configuring authorized networks. It is not possible to access the environment's cluster in Cloud Composer 3.

Change environment networking type

Console

  1. In the Google Cloud console, go to the Environments page.

    Go to Environments

  2. In the list of environments, click the name of your environment. The Environment details page opens.

  3. Go to the Environment configuration tab.

  4. In the Networking configuration section, find the Networking type item and click Edit.

  5. In the Networking type dialog, select:

    • Public IP environment (default) for Public IP networking.
    • Private IP environment for Private IP networking.

  6. Click Save.

gcloud

The Following Google Cloud CLI arguments change the environment's networking type:

  • --enable-private-environment: changes to Private IP networking.
  • --disable-private-environment: changes to Public IP networking (default).

Change to Private IP networking:

gcloud beta composer environments update ENVIRONMENT_NAME \
  --location LOCATION \
  --enable-private-environment

Change to Public IP networking:

gcloud beta composer environments update ENVIRONMENT_NAME \
  --location LOCATION \
  --disable-private-environment

Replace the following:

  • ENVIRONMENT_NAME: the name of the environment.
  • LOCATION: the region where the environment is located.

Example (Private IP):

gcloud beta composer environments update example-environment \
  --location us-central1 \
  --enable-private-environment

Example (Public IP):

gcloud beta composer environments update example-environment \
  --location us-central1 \
  --disable-private-environment

API

  1. Create an environments.patch API request.

  2. In this request:

    1. In the updateMask parameter, specify the config.private_environment_config.enable_private_environment mask.

    2. In the request body, in the enablePrivateEnvironment field:

      • Specify true to change to Private IP networking.
      • Specify false to change to Public IP networking (default).

Example (Private IP):

// PATCH https://composer.googleapis.com/v1beta1/projects/example-project/
// locations/us-central1/environments/example-environment?updateMask=
// config.private_environment_config.enable_private_environment

"config": {
  "privateEnvironmentConfig": {
    "enablePrivateEnvironment": true
  }
}

Terraform

The enable_private_environment field in the config block specifies the environment's networking type:

  • true: Private IP networking.
  • false or omitted: Public IP networking (default).
resource "google_composer_environment" "example" {
  provider = google-beta
  name = "ENVIRONMENT_NAME"
  region = "LOCATION"

  config {

    enable_private_environment = PRIVATE_IP_STATUS

  }
}

Replace the following:

  • ENVIRONMENT_NAME: the name of your environment.
  • LOCATION: the region where the environment is located.
  • PRIVATE_IP_STATUS: true for Private IP, false for Public IP

Example (Private IP):

resource "google_composer_environment" "example" {
  provider = google-beta
  name = "example-environment"
  region = "us-central1"

  config {

    enable_private_environment = true

    ... other configuration parameters
  }
}

Configure proxy server variables

You can set http_proxy and https_proxy environment variables in your environment. These standard Linux variables are used by web clients that run in containers of your environment's cluster to route traffic through the specified proxies.

The NO_PROXY variable by default is set to a list of Google domains and localhost so that they are excluded from proxying: .google.com,.googleapis.com,metadata.google.internal,localhost. This configuration makes it possible to create an environment with set http_proxy and https_proxy environment variables in cases when the proxy isn't configured to handle traffic to Google services.

What's next