gRPC direct connectivity

Direct connectivity is a connection solution that allows high-performance, authenticated, direct gRPC network connections between a Google Cloud client library and Cloud Storage, resulting in lower latency and connection overhead. When you use gRPC to connect to Google Cloud using direct connectivity, requests initiated through supported Google Cloud client libraries are routed directly to Cloud Storage, bypassing Google Front Ends (GFEs).

Direct connectivity is only available for requests made from Compute Engine virtual machines (VMs).

Requirements for direct connectivity

Direct connectivity is enabled by default when you use supported Cloud Storage client libraries to connect to Cloud Storage, but becomes available only if the following conditions are all met:

  • The Compute Engine VMs interacting with Cloud Storage must have an attached service account, even if the service account has no permissions. The service account is used to represent the Compute Engine VM in the Application Layer Transport Security handshake process.

  • The Compute Engine VMs interacting with a Cloud Storage bucket must be co-located with the bucket. For example, if the bucket is in us-central1, the VM can be located in us-central1-a.

  • Your routes and firewall rules allow IPv4 traffic to reach 34.126.0.0/18 and IPv6 traffic to reach 2001:4860:8040::/42. In addition, traffic must be allowed to reach the endpoints storage.googleapis.com:443 and directpath-pa.googleapis.com:443.

    To learn how to check whether a connection can be made to these endpoints, see Check network configurations. For information about setting up routes, see Configure routes.

Check connectivity by using the Google Cloud CLI

The Google Cloud CLI can test the diagnostics of direct connectivity on your bucket by uploading a 2-byte object to the bucket and then deleting it. To use this method, make sure you have the following IAM permissions on the bucket, which can be granted using the Storage Object User (roles/storage.objectUser) role:

  • storage.objects.create
  • storage.objects.delete
  • storage.objects.get

To check whether direct connectivity is available, run gcloud alpha storage diagnose with the argument --test-type=DIRECT_CONNECTIVITY:

gcloud alpha storage diagnose --test-type=DIRECT_CONNECTIVITY gs://BUCKET_NAME

Replace the following:

  • BUCKET_NAME: the name of your bucket. For example, my-bucket.

If a connection can be made, you receive a response similar to the following:

Using my-bucket bucket for the diagnostic tests.
Bucket location : US
Bucket storage class : STANDARD
Running diagnostic: Direct Connectivity Diagnostic...
WARNING: This diagnostic is experimental. The output may change, and checks may be added or removed at any time. Please do not rely on the diagnostic being present.
Finished running diagnostic: Direct Connectivity Diagnostic
Generating diagnostic report...
NAME
Direct Connectivity Diagnostic
    ┌────────────────────────────┬─────────────────────────────────────────────────────────────────────────┬──────────┐
    │            NAME            │                           PAYLOAD_DESCRIPTION                           │  RESULT  │
    ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────┼──────────┤
    │ Direct Connectivity Upload │ Able to upload object to bucket using Direct Connectivity network path. │ Success. │
    └────────────────────────────┴─────────────────────────────────────────────────────────────────────────┴──────────┘

Check network configurations

If you encounter an Unable to connect to Traffic Director error while checking for direct connectivity using the Google Cloud CLI, use the following instructions to ensure that your network configurations allow traffic to pass from your VMs to the required endpoints.

Direct connectivity requires a service hosted at directpath-pa.googleapis.com:443 and at storage.googleapis.com:443. To check whether a connection can be made to the service, make a curl call to directpath-pa.googleapis.com:443 or storage.googleapis.com:443. For example:

curl directpath-pa.googleapis.com:443

If a connection can be made, you receive an empty response without errors:

curl: (52) Empty reply from server

If a connection cannot be made, you receive an error similar to the following:

curl: (56) Recv failure: Connection reset by peer