Update your projects to use zonal DNS

This document explains how to migrate an existing project from global DNS to zonal DNS. Zonal DNS enhances reliability by isolating outages within zones, preventing disruptions to essential services like instance creation and autohealing.

Before you begin

• If you haven't already, then set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:
  

  Select the tab for how you plan to use the samples on this page:

  Console

  When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

  gcloud

  1. Install the Google Cloud CLI, then initialize it by running the following command:

     gcloud init

  2. Set a default region and zone.

  REST

  To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

  Install the Google Cloud CLI, then initialize it by running the following command:

  gcloud init

  For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Required roles

To get the permissions that you need to migrate projects to use zonal DNS, ask your administrator to grant you the following IAM roles:

• Migrate a project to use zonal DNS: Project Editor (roles/resourcemanager.projectEditor) on the project
• Migrate VMs to zonal DNS within a project: Compute Instance Admin (v1) (roles/compute.instanceAdmin.v1) on the project
• If your VM uses a service account: Service Account User (roles/iam.serviceAccountUser) on the service account or project

For more information about granting roles, see Manage access to projects, folders, and organizations.

These predefined roles contain the permissions required to migrate projects to use zonal DNS. To see the exact permissions that are required, expand the Required permissions section:

You might also be able to get these permissions with custom roles or other predefined roles.

Migrate your project to zonal DNS

To migrate a project to use zonal DNS, complete the following tasks:

1. Check if your project uses global DNS by default
2. Determine the migration readiness of your projects by using query analysis
3. Migrate projects that are compatible with zonal DNS
4. Fix incompatible queries
5. Monitor global DNS logs to confirm migration readiness.
6. Migrate the remaining projects to zonal DNS
7. Check if a change to zonal DNS is impacting your project

Check if your project uses global DNS by default

Check your projects to see if they need to migrate from using global DNS to zonal DNS. You only need to migrate projects that are configured to use global DNS as the default for any internal DNS names created within the project.

gcloud compute project-info describe --project=PROJECT_ID --flatten="vmDnsSetting"

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID?fields=id,name,vmDnsSetting

Use query analysis to determine the migration readiness of a project

To assess if a project can be migrated to zonal DNS without code changes or altering how global DNS is used, Google Cloud analyzes your DNS query history. This analysis provides the following metrics that indicate project compatibility with zonal DNS:

• zonal_dns_ready (Compatible queries): This metric represents the total number of queries within a 100-day period that can be successfully resolved using zonal DNS.

• zonal_dns_risky (Incompatible queries): This metric represents the total number of queries that can't be resolved using zonal DNS. These queries typically involve cross-region communication or other scenarios where zonal resolution fails. Crucially, if this metric has a non-zero value, your project is not yet ready for migration. You'll must address these incompatible queries before switching to zonal DNS.

To view these metrics, use the Metrics Explorer in the Google Cloud console.

1. In the Google Cloud console, go to the leaderboard Metrics explorer page:

   Go to Metrics explorer

   If you use the search bar to find this page, then select the result whose subheading is Monitoring.

2. On the right side of the toolbar that contains the Select a metric field, click Code editor, MQL, or PromQL.

3. If the query input field isn't titled MQL Query, then in the lower right corner of the query input field, for Language, select MQL.

4. In the query input field, enter the following text exactly as it appears:

   fetch compute.googleapis.com/Location
   | metric 'compute.googleapis.com/global_dns/request_count'
   | every 1d
   | within 7d
   

5. Click the Run query button.

   The Google Cloud console displays a chart of the two metrics (zonal_dns_ready and zonal_dns_risky) and the corresponding number of queries made over the time period for each metric.

   

6. Check the value for the zonal_dns_risky metric.

   • If the value is 0, then the project is ready for migration to zonal DNS. You can migrate the project, as described in Migrate projects that are ready to zonal DNS.
   • If the value is a non-zero number, such as 0.02k as shown in the previous screenshot, then there are some queries that might not work after you migrate to zonal DNS. The project is not ready for migration. Continue with the steps in Fix incompatible queries.

Migrate projects that are compatible with zonal DNS

Use any of the following options to migrate projects that are ready to switch to zonal DNS:

• Click the Use Zonal DNS button in the Google Cloud console.

  1. When you view the VM instances page for your project, if your project is ready for migration (is compatible with zonal DNS queries), then the banner includes a recommendation to Use Zonal DNS. This recommendation is based on internal DNS usage in the project, but is limited to the last 30 days.

     

     If you click the Use Zonal DNS button, then the project metadata is updated to use zonal DNS.

  2. Optional: View and query VM metadata to confirm the metadata change.

• Manually change the project metadata to use zonal DNS.

  1. Enable zonal DNS for your instances by setting the vmDnsSetting metadata entry for the project. After you set this metadata entry, your compute instances can be accessed only by their zonal DNS names (VM_NAME.ZONE.c.PROJECT_ID.internal) when using search paths. The instances still retain both the zonal and global search paths, but their global DNS names, which don't include the ZONE as part of the internal DNS name, no longer function. Only instances in the same region and same project can access each other using the global name when this setting is in place.

     Console

     1. To update the setting at the project level, in the Google Cloud console, go to the Compute Engine Metadata page.

        Go to the Custom metadata page

     2. Click edit Edit.

     3. If a key with the value VmDnsSetting exists, change its value to ZonalOnly.

     4. If a key with the value VmDnsSetting doesn't exist, then click add_box Add item.

        • In the Key field, enter VmDnsSetting.
        • In the Value field, enter ZonalOnly.

     5. To finish modifying your custom metadata entries, click Save.

     gcloud

     1. To update the metadata setting for the current project, use the project-info add-metadata command.

        gcloud compute project-info add-metadata \
            --metadata vmDnsSetting=ZonalOnly
        

     2. Optional: To verify the metadata settings for a project, use the following command:

        gcloud compute project-info describe --project=PROJECT_ID --flatten="vmDnsSetting"
        

        Replace PROJECT_ID with the name of the project to query.

     REST

     To update the metadata setting at the project level, construct a POST request using the projects.setCommonInstanceMetadata method.

     1. Optional: To perform optimistic locking, you can optionally provide a fingerprint.

        A fingerprint is a random string of characters generated by Compute Engine. The fingerprint changes after each request, and if you provide a mismatched fingerprint, your request is rejected.

        If you don't provide a fingerprint, no check for consistency is performed, and the projects.setCommonInstanceMetadata request succeeds. If you use the instances.setMetadata method, then a fingerprint is always required.

        To get the current fingerprint of a project, call the project.get method.

        GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
        

        The output is similar to the following:

        {
         "name": "myproject",
         "commonInstanceMetadata": {
           "kind": "compute#metadata",
           "fingerprint": "FikclA7UBC0=",
           ...
         }
        }
        

     2. Construct a POST request to the projects.setCommonInstanceMetadata method to set the metadata key-value pair:

        POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/setCommonInstanceMetadata
        
        {
        "fingerprint": "FikclA7UBC0=",
        "items": [
          {
          "key": "vmDnsSetting",
          "value": "ZonalOnly"
          }
        ]
        }
        

        Replace PROJECT_ID with your your project ID.

  2. After you configure the vmDnsSetting metadata entry for your project, refresh the DHCP lease on each instance in that project. You can refresh the lease by restarting the instance, or by waiting for the lease to expire, or by running one of the following commands:

     Linux instances

     sudo dhclient -v -r
     

     Windows instances

     ipconfig /renew
     

  3. Check that your project uses zonal DNS.

Fix incompatible queries

A project that is not ready to migrate means that at least one incompatible DNS query was made within a certain amount of time, such as the last 30 days. Incompatible queries might have the following attributes:

• Make a call to a compute instance in a different project
• Make a call to a compute instance in a different region

If your project has incompatible queries, the following banner appears in the VM instances page of the Google Cloud console:

To fix all incompatible queries, we recommend that you use the zonal fully qualified domain name (FQDN) of the source instance in the query. This approach ensures query resolution remains undisrupted after migrating the project to zonal DNS.

To resolve incompatible queries, do the following:

1. Use the Logs Explorer to access and query the global DNS usage for compute instances your project.

   Go to Logs Explorer

2. Select the project.

3. Apply the resource and log name filters:

   1. Click Resource.
   2. In the Select resource dialog, select VM Instance, and then click Apply.
   3. Click Log name.

   4. In the Select log names dialog, select gdnsusage, and then click Apply.

   Alternatively, you can enter the following into the query field:

      resource.type="gce_instance"
      log_name="projects/PROJECT_ID/logs/compute.googleapis.com%2Fgdnsusage"
     

4. In the Query results pane, each query has a jsonPayload field. Each jsonPayload field contains the following information:

   • The source VM name, its project ID, and zone name.
   • The destination VM name, its project ID, and zone name.

   • A debug message that provides information about how to update the global DNS query that can't be resolved by using zonal DNS names. These are considered migration blocking queries that you should debug and fix.

     "To use Zonal DNS, update the Global DNS query sent from the source VM
     VM_NAME.c.PROJECT_ID.internal to the following zonal
     FQDN: VM_NAME.ZONE.c.PROJECT_ID.internal"
     

   • A query count that shows how many migration blocking queries the source VM sends to the destination VM for that day.

   The following screenshot shows the jsonPayload field information in the Logs Explorer console page.

   

5. Use the information in the jsonPayload that you obtained in the previous step to determine what FQDN to use to manually update your global DNS queries to use zonal DNS, and make the project ready for migration. The most common use cases for where to update the FQDN and resolve compatibility are as follows:

   • Internal DNS names from the metadata server: No action is required because the returned DNS name will change to a zonal FQDN immediately after the migration to zonal DNS. If the DNS name is cached, you just need to make one more call to update the cache value.
   • Internal DNS names used to access VMs in another region: If you have an application that uses the internal DNS names for VMs in different regions, you can modify the DHCP policy or configuration file to include the zone in the other region.
   • Hard-coded global FQDN: If you have an application that uses hard-coded global FQDN names for VMs, you can update the call within the application to use the internal DNS name or zonal FQDN instead. You can make this change through a code change or configuration change in Terraform.
   • VMs in service projects that use a Shared VPC network: To resolve DNS names of VMs in service projects that use a Shared VPC network, you must use the zonal FQDNs of the VMs.

6. After you have updated the global DNS queries to use zonal DNS, do the following:

   1. Use the Logs Explorer page to query the global DNS usage again. After you fix all blocking global DNS queries, there should be no debug logs displayed in the query results.

   2. Recheck the monitoring metric to see if all incompatible DNS queries have been removed.

View global DNS logs in Logs Explorer

Logs Explorer primarily shows global DNS logs for projects with queries that are incompatible with zonal DNS. These logs help you to identify and analyze those problematic queries before migrating.

You can also utilize Logs Explorer for these incompatible queries to do the following:

• Create dashboards: Visualize your incompatible global DNS query patterns to gain insights into your application's communication behavior.
• Aggregate logs: Analyze DNS logs across your entire organization to identify broader trends and potential areas for improvement.

Check if a change to zonal DNS is impacting your project

After migrating to zonal DNS, it's crucial to verify that your applications and services are still working correctly. Since zonal DNS changes how internal DNS names are resolved, some applications might experience issues if they rely on global DNS names.

The following section describes how to check for potential impacts and how to resolve them:

1. Command-line instance communication

   Task: Try pinging one instance from another instance using gcloud CLI.

   gcloud compute ssh VM-A --command "ping VM-B"
   

   Potential Error: "Could not resolve host"—This means VM-A can't find the IP address for VM-B.

   Resolution: Update the hostname you're using for VM-B to its fully qualified domain name (FQDN), which includes the zone name: INSTANCE_NAME.ZONE.c.PROJECT_ID.internal

2. Instance communication within Compute Engine services

   Task: If you're using health checks for managed instance groups (MIGs) that rely on internal DNS names, check if the health checks are passing.

   Potential Error: "Health check failed"—This indicates the health check can't reach its target due to DNS resolution problems.

   Resolution: Ensure the health check is using the FQDN of the target instance, including the zone name.

3. Application-specific use cases

   Many applications rely on internal DNS for tasks like the following:

   • Connecting to databases (for example, Cloud SQL)

   • Interacting with message queues (for example, Pub/Sub)

   • Potential Errors: These vary depending on the application but might include:

     • "Unable to connect to SERVICE_NAME"
     • "Connection timed out"
     • "No such host is known"

   • Resolution: Check your application's configuration to make sure it's using the FQDN (including the zone name) when referencing services.

Revert to using global DNS

You can undo the migration to zonal DNS by changing the default internal DNS type back to global DNS. You can do this at the organization, project, instance, or container level.

Revert to using global DNS for a project

To revert a project to using global DNS, complete the following steps.

1. Add the following to the project's metadata: vmDnsSetting=GlobalDefault.

   For information about how to set project metadata values, see Set and remove custom metadata.

2. Verify that none of the instances in the project have the vmDnsSetting metadata value set to ZonalOnly.

   gcloud compute instances describe INSTANCE_NAME --flatten="metadata[]"
   

   Replace INSTANCE_NAME with the name of the instance to check.

3. Refresh the DHCP lease on each instance. You can refresh the lease by restarting the instance, waiting for the lease to expire, or by running one of the following commands in the guest operating system:

   • Linux instances: sudo dhclient -v -r
   • Windows Server instance: ipconfig /renew

Revert to using global DNS for an instance

To revert a specific instance to using global DNS, complete the following steps.

1. Update the instance's metadata to include vmDnsSetting=GlobalDefault.

   For information about how to set compute instance metadata values, see Set and remove custom metadata.

2. To force the DNS configuration change, restart the network of the instance using one of the following commands:

   • For Container-Optimized OS or Ubuntu:

     sudo systemctl restart systemd-networkd
     

   • For CentOS, RedHat EL, Fedora CoreOS, or Rocky Linux:

     sudo systemctl restart network
     

     or

     sudo systemctl restart NetworkManager.service
     

   • For Debian:

     sudo systemctl restart networking
     

   • For Linux systems with nmcli:

     sudo nmcli networking off
     sudo nmcli networking on
     

   • For Windows:

     ipconfig /renew
     

Revert to using global DNS for a container

If you run your application or workload in containers, on Google Kubernetes Engine, or on App Engine flexible environment, the DNS configuration in your container settings might not automatically update until you restart the containers. To disable zonal DNS on these container apps, complete the following steps.

1. Set the project metadata setting vmDnsSetting to GlobalDefault on the projects that own the containers and VMs.

2. Restart the containers so that their DNS settings revert to the original state.

Troubleshoot the global DNS to zonal DNS migration process

If you have issues with the migration process, refer to the troubleshooting guide.

What's next

• Review the Google Cloud resource hierarchy for information about the relationship between organizations, folders, and projects.
• Learn more about internal DNS for Compute Engine.