API requests fail with TARGET_CONNECT_HOST_NOT_REACHABLE error

You're viewing Apigee and Apigee hybrid documentation.
There is no equivalent Apigee Edge documentation for this topic.

Symptoms

API requests fail with TARGET_CONNECT_HOST_NOT_REACHABLE error.

Error Messages

If this issue occurs, the API requests will fail with HTTP 503 response status code and the following error:

{"fault":{"faultstring":
"Unable to resolve host invalid-target-host","detail":
{"errorcode":"protocol.http.NoResolvedHost","reason":
"TARGET_CONNECT_HOST_NOT_REACHABLE"}}}

Possible Causes

The following potential causes have been identified for the aforementioned symptom:

Cause	Description	Platform
The target server host specified is incorrect or has invalid characters	This issue could occur in the event that the designated target server host specified within the API proxy is either incorrect or contains invalid characters.	Apigee, Apigee hybrid
DNS peering is not configured	This issue could occur when Apigee is not able to resolve the domain name if DNS peering is not configured in Apigee deployments.	Apigee

Cause: target server host specified is incorrect or has invalid characters

Diagnosis

Send an API request to the relevant API proxy:

curl -ik https://dev.example.com/dns-peering-example
  HTTP/2 503
  content-type: application/json
  x-request-id: ***
  content-length: 169
  date: Thu, 02 Nov 2023 04:31:43 GMT
  via: 1.1 google
  alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

and check the response message:

{"fault":{"faultstring":
"Unable to resolve host invalid-target-host","detail":
{"errorcode":"protocol.http.NoResolvedHost","reason":
"TARGET_CONNECT_HOST_NOT_REACHABLE"}}}

If the response contains the error reason TARGET_CONNECT_HOST_NOT_REACHABLE then, that is related to this reason.

Resolution

Check the API proxy definition and find the target hostname defined:
If the target hostname given is invalid or has invalid characters, fix it accordingly, create a new revision of the proxy, and deploy the proxy.

Cause: DNS peering is not configured

Diagnosis

Check whether the Apigee organization is peered with a VPC network by invoking the following Apigee API:
```
TOKEN=$(gcloud auth print-access-token)
```
```
curl -H "Authorization: Bearer $TOKEN" "https://apigee.googleapis.com/v1/organizations/$ORG" | jq .authorizedNetwork
```
For example:
```
TOKEN=$(gcloud auth print-access-token)
```
```
curl -H "Authorization: Bearer $TOKEN" "https://apigee.googleapis.com/v1/organizations/example-org/" | jq .authorizedNetwork
```
Response:
```
"projects/example-org/global/networks/shared-vpc1"
```
If the Apigee organization is not peered with a VPC and uses Private Service Connect (PSC), then, this scenario does not apply for this Apigee organization.

Note: As of January, 2024, private DNS peering is not supported in non-VPC peering Apigee deployments. The non-VPC peering Apigee deployments could still use publicly resolvable DNS names for PSC endpoint attachments. Hence, this scenario will only apply to VPC peered Apigee deployments using internal target endpoints.
Verify with the API proxy developer from the customer side whether this target server domain name is internally configured. If not, this scenario does not apply.
Find the project ID and the network in which the target endpoint is hosted.
List DNS peerings created in the above network by executing the peered-dns-domains list command:
```
gcloud services peered-dns-domains list --network=NETWORK --project=PROJECT-ID
```
The above command could either return a blank result if no peered DNS domains are available, or list peered DNS domains similar to following:
```
NAME                 DNS_SUFFIX
customer-service     customer.service.internal.
accounts-service     accounts.service.internal.
```
If the above command does not list a DNS peering entry for the relevant DNS suffix, then, that could be the reason for this issue. Follow the instructions given in Resolution to resolve it.

Resolution

Make a note of the DNS suffix, project ID, and network in which the target endpoint is hosted.

Create a peered DNS domain for the DNS suffix using the


          peered-dns-domains create

gcloud command. Note that the DNS suffix should contain a trailing dot at the end of the DNS suffix.

gcloud services peered-dns-domains create NAME --network=NETWORK --dns-suffix=DNS-SUFFIX. --project=PROJECT-ID

For example:

gcloud services peered-dns-domains create orders-service --network="shared-vpc1" --dns-suffix="orders.service.internal." --project=service-project

Response:

Operation "operations/cpdd.p25-1064980322781-fafa5fe4-b5fe-487e-830d-fff0f9a6200d" finished successfully.

Now, send an API request to the API proxy endpoint and verify whether the API proxy could resolve the target server domain name and communicate with the target server:

gcloud services peered-dns-domains create orders-service --network="shared-vpc1" --dns-suffix="orders.service.internal." --project=service-project

Response:

Operation "operations/cpdd.p25-1064980322781-fafa5fe4-b5fe-487e-830d-fff0f9a6200d" finished successfully

Must gather diagnostic information

If the problem persists even after following the above instructions, gather the following diagnostic information, and then contact Google Cloud Customer Care.

Google Cloud project ID
Apigee organization
API proxy and revision
Network in which the private domain is created
DNS suffix of the private domain
The complete output of the peered DNS domain creation command