Manage private connectivity configurations

Stay organized with collections Save and categorize content based on your preferences.

In this page, you learn how to use the Datastream API to create, retrieve information about, and delete private connectivity configurations.

Private connectivity configurations contain information that Datastream uses to communicate with a data source over a private network (internally within Google Cloud, or with external sources connected over VPN or Interconnect). This communication happens through a Private Service Connect interface or a Virtual Private Cloud (VPC) peering connection.

Create a private connectivity configuration using VPC peering

Before creating a private connectivity configuration, make sure that you:

• Have a VPC network to peer to Datastream's private network. For more information about creating a VPC network, see Using VPC Network Peering.
• Have an available IP range on the VPC network. Datastream uses this IP range to create VPC peering with the provided VPC, so the range can't overlap with subnets within that VPC. As a result, Datastream can communicate with the source.

There are two parameters associated with creating a private connectivity configuration:

• vpcName: Specify the name and path of the VPC network.
• subnet: Provide an available IP range on the VPC network.

The following code shows a request to create a private connectivity configuration, and specifies the connectivity method as using VPC peering (vpcPeeringConfig).

POST https://datastream.googleapis.com/v1/projects/PROJECT_ID/locations/
LOCATION/privateConnections?privateConnectionId=PRIVATE_CONNECTIVITY_CONFIGURATION_ID
{
  "displayName": "DISPLAY_NAME",
  "vpcPeeringConfig": {
      "vpcName": "VPC_NAME",
      "subnet": "SUBNET"
  }
}

POST https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections?privateConnectionId=myPrivateConnection
{
  "displayName": "my-private-connection",
  "vpcPeeringConfig": {
    "vpcName": "projects/my-project/global/networks/my-vpc",
    "subnet": "10.11.12.13/29"
  }
}

{
  "name": "projects/projectId/location/us-central1/privateConnections/myPrivateConnection",
  "createTime": "2019-12-22T16:17:37.159786963Z",
  "updateTime": "2019-12-22T16:17:37.159786963Z",
  "displayName": "my-private-connection",
  "state": "CREATED",
  "vpcPeeringConfig": {
    "vpcName": "projects/my-project/global/networks/my-vpc",
    "subnet": "10.11.12.13/29"
  }  
}

gcloud datastream private-connections create PRIVATE_CONNECTION_NAME --location=LOCATION --display-name=DISPLAY_NAME --vpc=VPC_NAME --subnet=SUBNET

gcloud datastream private-connections create my-privateConnection --location=us-central1 --display-name=my-private-connection --vpc=my-vpc --subnet=10.11.12.13/29

Create a private connectivity configuration using a Private Service Connect interface

Before creating a private connectivity configuration, make sure that you:

• Have a VPC network to connect to the Datastream private network. For more information about creating a VPC network, see Create and manage VPC networks.

• Create a network attachment in your VPC project.

• Verify that Google Cloud and the on-premises firewall allow traffic from the network attachment IP address range to the source database from which you want to stream data.

The following code shows a request to create a private connectivity configuration using Private Service Connect interfaces. You first need to run a command to create a private connectivity configuration with the validateOnly flag (--validate-only for Google Cloud CLI):

POST "https://datastream.googleapis.com/v1/projects/PROJECT_ID/locations/
LOCATION/privateConnections?privateConnectionId=PRIVATE_CONNECTIVITY_CONFIGURATION_ID&validateOnly=true"

gcloud datastream private-connections create PRIVATE_CONNECTION_NAME --location=LOCATION --display-name=DISPLAY_NAME --networkAttachment=NETWORK_ATTACHMENT_NAME --validate-only

In the response, you get the name of the producer project to approve. You need to add it to your network attachment allowed projects list. Once added, run the command to create the private connectivity configuration again, this time without the validateOnly flag:

POST https://datastream.googleapis.com/v1/projects/PROJECT_ID/locations/
LOCATION/privateConnections?privateConnectionId=PRIVATE_CONNECTIVITY_CONFIGURATION_ID
{
  "displayName": "DISPLAY_NAME",
  "pscInterfaceConfig": {
      "networkAttachment": "NETWORK_ATTACHMENT_NAME"
  }
}

POST https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections?privateConnectionId=myPrivateConnection
{
  "displayName": "my-private-connection",
  "pscInterfaceConfig": {
    "networkAttachment": "projects/my-project/regions/us-central1/networkAttachments/my-na"
  }
}

gcloud datastream private-connections create PRIVATE_CONNECTION_NAME --location=LOCATION --display-name=DISPLAY_NAME --networkAttachment=NETWORK_ATTACHMENT_NAME

gcloud datastream private-connections create my-privateConnection --location=us-central1 --display-name=my-private-connection --networkAttachment=my-network-attachment

Get information about a private connectivity configuration

The following code shows a request to retrieve information about a private connectivity configuration that has been created.

GET https://datastream.googleapis.com/v1/projects/PROJECT_ID/locations/
LOCATION/privateConnections/PRIVATE_CONNECTIVITY_CONFIGURATION_ID

GET https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections/myPrivateConnection
     

List private connectivity configurations

The following code shows a request to retrieve information about all of your private connectivity configurations.

GET https://datastream.googleapis.com/v1/projects/PROJECT_ID/locations/
LOCATION/privateConnections

Delete a private connectivity configuration

The following code shows a request to delete a private connectivity configuration. After the configuration is deleted, any connection profiles using the configuration won't work. In addition, any streams using these connection profiles won't be operable.

To resolve this issue, assign a different private connectivity configuration to each connection profile. For more information, see Update a connection profile.

DELETE https://datastream.googleapis.com/v1/projects/PROJECT_ID/locations/
LOCATION/privateConnections/[private-connectivity-configuration-id] {"force": true} 

DELETE https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections/myPrivateConnection {"force": true} 

What's next

• Learn more about private connectivity.
• Learn about other network connectivity options you can use with Datastream.
• Learn how to use the Datastream API to manage connection profiles.
• Learn how to use the Datastream API to manage streams.
• For more information about using the Datastream API, see the reference documentation.