API-Proxy-Bereitstellungen schlagen fehl, weil apigee-serving-cert nicht gefunden wurde oder abgelaufen ist
Mit Sammlungen den Überblick behalten
Sie können Inhalte basierend auf Ihren Einstellungen speichern und kategorisieren.
Sie lesen gerade die Dokumentation zu Apigee und Apigee Hybrid. Apigee Edge-Dokumentation aufrufen.
Symptome
API-Proxy-Bereitstellungen schlagen mit den folgenden Fehlermeldungen fehl.
Fehlermeldungen
Wenn das TLS-Zertifikat des Dienstes apigee-webhook-service.apigee-system.svc abgelaufen oder noch nicht gültig ist, wird die folgende Fehlermeldung in Logs vom Typ apigee-watcher angezeigt:
Wenn doppelte Zertifikatsanfragen zur Verlängerung des apigee-serving-cert-Zertifikats erstellt werden, wird das apigee-serving-cert-Zertifikat möglicherweise nicht verlängert.
Wenn cert-manager nicht fehlerfrei ist, wird das Zertifikat apigee-serving-cert möglicherweise nicht verlängert.
Ursache: Das Apigee-Bereitstellungszertifikat wurde nicht gefunden
Diagnose
Prüfen Sie die Verfügbarkeit des Zertifikats apigee-serving-cert im Namespace apigee-system:
kubectl -n apigee-system get certificates apigee-serving-cert
Wenn dieses Zertifikat verfügbar ist, sollte die Ausgabe in etwa so aussehen:
NAME READY SECRET AGE
apigee-serving-cert True webhook-server-cert 2d10h
Wenn das Apigee-Serving-Zertifikat nicht im Namespace apigee-system gefunden wird, kann dies der Grund für das Problem sein.
Lösung
apigee-serving-cert wird während der Installation von Apigee Hybrid durch den Befehl apigeectl
init erstellt. Führen Sie daher diesen Befehl mit der entsprechenden Datei overrides.yaml aus, um sie neu zu erstellen:
apigeectl init -f overrides/overrides.yaml
Prüfen Sie, ob das Zertifikat apigee-serving-cert erstellt wurde:
kubectl -n apigee-system get certificates apigee-serving-cert
Ursache: Doppelte Zertifikatsanfragen wurden zur Verlängerung von apigee-serving-cert erstellt
Diagnose
Prüfen Sie die cert-manager-Controller-Logs und prüfen Sie, ob eine Fehlermeldung ähnlich der folgenden zurückgegeben wurde.
Listen Sie alle cert-manager-Pods auf:
kubectl -n cert-manager get pods
Beispielausgabe:
NAME READY STATUS RESTARTS AGE
cert-manager-66d9545484-772cr 1/1 Running 0 6d19h
cert-manager-cainjector-7d8b6bd6fb-fpz6r 1/1 Running 0 6d19h
cert-manager-webhook-669b96dcfd-6mnm2 1/1 Running 0 6d19h
Prüfen Sie die cert-manager-Controller-Logs:
kubectl -n cert-manager logs cert-manager-66d9545484-772cr | grep "issuance is skipped until there are no more duplicates"
Beispielausgabe:
1controller.go:163]cert-manager/certificates-readiness"msg"="re-queuing item due to error processing""error"="multiple CertificateRequests were found for the 'next' revision 3, issuance is skipped until there are no more duplicates""key"="apigee-system/apigee-serving-cert"
Wenn eine Fehlermeldung wie diese angezeigt wird, wird das apigee-serving-cert-Zertifikat nicht verlängert.
Listen Sie alle Zertifikatsanfragen im Namespace apigee-system auf und prüfen Sie, ob mehrere Zertifikatsanfragen für die Verlängerung derselben apigee-serving-cert-Zertifikatsüberarbeitung erstellt wurden:
Prüfen Sie, ob doppelte Zertifikatsanfragen gelöscht wurden und nur eine Zertifikatsanfrage für das Zertifikat apigee-serving-cert im Namespace apigee-system verfügbar ist:
kubectl -n apigee-system get certificaterequests
Prüfen Sie, ob das Zertifikat apigee-serving-cert verlängert wurde:
kubectl -n apigee-system get certificates apigee-serving-cert -o yaml
Prüfen Sie den Status der cert-manager-Pods im Namespace cert-manager:
kubectl -n cert-manager get pods
Wenn cert-manager-Pods fehlerfrei sind, sollten alle cert-manager-Pods (1/1) und im Status Running bereit sein. Andernfalls kann dies der Grund für das Problem sein:
NAME READY STATUS RESTARTS AGE
cert-manager-59cf78f685-mlkvx 1/1 Running 0 15d
cert-manager-cainjector-78cc865768-krjcp 1/1 Running 0 15d
cert-manager-webhook-77c4fb46b6-7g9g6 1/1 Running 0 15d
Der cert-manager kann aus verschiedenen Gründen fehlschlagen. Prüfen Sie die cert-manager-Logs, identifizieren Sie den Grund für den Fehler und beheben Sie sie entsprechend.
Ein bekannter Grund ist, dass cert-manager fehlschlägt, wenn er nicht mit der Kubernetes API kommunizieren kann. In diesem Fall erhalten Sie eine Fehlermeldung ähnlich der folgenden:
Prüfen Sie den Status des Kubernetes-Clusters und beheben Sie etwaige Probleme.
Siehe
Fehlerbehebung für Cluster.
Weitere Informationen
Fehlerbehebung für zusätzlichecert-manager Informationen zur Fehlerbehebung.
Erfassen von Diagnoseinformationen erforderlich
Wenn das Problem auch nach Befolgen der obigen Anweisungen weiterhin besteht, sammeln Sie die folgenden Diagnoseinformationen und wenden Sie sich dann an den Google Cloud Customer Care:
Google Cloud-Projekt-ID
Apigee Hybrid-Organisation
Apigee Hybrid-overrides.yaml-Datei, in der alle vertraulichen Informationen maskiert sind.
Kubernetes-Pod-Status in allen Namespaces:
kubectl get pods -A > kubectl-pod-status`date +%Y.%m.%d_%H.%M.%S`.txt
Kubernetes-Dump cluster-info:
# generate kubernetes cluster-info dump
kubectl cluster-info dump -A --output-directory=/tmp/kubectl-cluster-info-dump
# zip kubernetes cluster-info dump
zip -r kubectl-cluster-info-dump`date +%Y.%m.%d_%H.%M.%S`.zip /tmp/kubectl-cluster-info-dump/*
[[["Leicht verständlich","easyToUnderstand","thumb-up"],["Mein Problem wurde gelöst","solvedMyProblem","thumb-up"],["Sonstiges","otherUp","thumb-up"]],[["Schwer verständlich","hardToUnderstand","thumb-down"],["Informationen oder Beispielcode falsch","incorrectInformationOrSampleCode","thumb-down"],["Benötigte Informationen/Beispiele nicht gefunden","missingTheInformationSamplesINeed","thumb-down"],["Problem mit der Übersetzung","translationIssue","thumb-down"],["Sonstiges","otherDown","thumb-down"]],["Zuletzt aktualisiert: 2025-08-28 (UTC)."],[[["\u003cp\u003eAPI proxy deployments in Apigee hybrid may fail due to an expired or invalid TLS certificate for the \u003ccode\u003eapigee-webhook-service.apigee-system.svc\u003c/code\u003e, leading to errors in \u003ccode\u003eapigee-watcher\u003c/code\u003e logs.\u003c/p\u003e\n"],["\u003cp\u003eThe issue can be caused by the \u003ccode\u003eapigee-serving-cert\u003c/code\u003e not being found, duplicate certificate requests, or an unhealthy \u003ccode\u003ecert-manager\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eResolutions involve updating the \u003ccode\u003eapigee-serving-cert\u003c/code\u003e using Helm, deleting duplicate certificate requests in the \u003ccode\u003eapigee-system\u003c/code\u003e namespace, and ensuring the \u003ccode\u003ecert-manager\u003c/code\u003e is healthy.\u003c/p\u003e\n"],["\u003cp\u003eIf problems persist, diagnostic information such as Google Cloud Project ID, Apigee hybrid organization details, \u003ccode\u003eoverrides.yaml\u003c/code\u003e file, Kubernetes pod status, and a Kubernetes \u003ccode\u003ecluster-info\u003c/code\u003e dump should be gathered for Google Cloud Customer Care.\u003c/p\u003e\n"]]],[],null,["*You're viewing **Apigee** and **Apigee hybrid** documentation.\nView [Apigee Edge](https://docs.apigee.com/api-platform/troubleshoot/support-d) documentation.*\n| **Note:** This document is applicable for Apigee hybrid users only.\n\nSymptoms\n\n\nAPI proxy deployments fail with the following error messages.\n\nError Messages\n\n\nIf the TLS certificate of the\n`apigee-webhook-service.apigee-system.svc` service has expired\nor is not yet valid, the following error message will be shown on\n`apigee-watcher` logs: \n\n```transact-sql\n{\"level\":\"error\",\"ts\":1687991930.7745812,\"caller\":\"watcher/watcher.go:60\",\n\"msg\":\"error during watch\",\"name\":\"ingress\",\"error\":\"INTERNAL: INTERNAL: failed\nto update ApigeeRoute [org-env]-group-84a6bb5, namespace apigee:\nInternal error occurred: failed calling webhook\n\\\"mapigeeroute.apigee.cloud.google.com\\\": Post\n\\\"https://apigee-webhook-service.apigee-system.svc:443/mutate-apigee-cloud-google-com-v1alpha1-apigeeroute?timeout=30s\\\":\nx509:\ncertificate has expired or is not yet valid: current time\n2023-06-28T22:38:50Z is after 2023-06-17T17:14:13Z, INTERNAL: failed to update\nApigeeRoute [org-env]-group-e7b3ff6, namespace apigee \n```\n\nPossible Causes\n\n| Cause | Description |\n|----------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| [The apigee-serving-cert is not found](#not-found) | If the `apigee-serving-cert` is not found in the `apigee-system` namespace, this issue could occur. |\n| [Duplicate certificate requests were created for renewing `apigee-serving-cert`](#duplicate) | If there are duplicate certificate requests created for renewing the `apigee-serving-cert` certificate, the `apigee-serving-cert` certificate may not get renewed. |\n| [cert-manager is not healthy](#not-healthy) | If `cert-manager` is not healthy, the `apigee-serving-cert` certificate may not get renewed. |\n\nCause: The apigee-serving-cert is not found\n\nDiagnosis\n\n1.\n Check the availability of the `apigee-serving-cert` certificate\n in the `apigee-system` namespace:\n\n ```\n kubectl -n apigee-system get certificates apigee-serving-cert\n ```\n\n\n If this certificate is available, an output similar to following should\n be seen: \n\n ```text\n NAME READY SECRET AGE\n apigee-serving-cert True webhook-server-cert 2d10h\n ```\n2. If the apigee-serving-cert certificate is not found in the `apigee-system` namespace, that could be the reason for this issue.\n\nResolution\n\n1. Update the `apigee-serving-cert` using [Helm](/apigee/docs/hybrid/latest/helm-reference): \n\n ```\n helm upgrade ENV_NAME apigee-env/ \\\n --namespace APIGEE_NAMESPACE \\\n --set env=ENV_NAME \\\n --atomic \\\n -f OVERRIDES_FILE\n ```\n\n Make sure to include all of the settings shown, including `--atomic`\n so that the action rolls back on failure.\n2. Verify that the `apigee-serving-cert` certificate has been created: \n\n ```\n kubectl -n apigee-system get certificates apigee-serving-cert\n ```\n\nCause: Duplicate certificate requests were created for\nrenewing apigee-serving-cert\n\nDiagnosis\n\n1.\n Check `cert-manager` controller logs and see whether an error\n message similar to the following has been returned.\n\n List all `cert-manager` pods: \n\n ```\n kubectl -n cert-manager get pods\n ```\n\n An example output: \n\n ```text\n NAME READY STATUS RESTARTS AGE\n cert-manager-66d9545484-772cr 1/1 Running 0 6d19h\n cert-manager-cainjector-7d8b6bd6fb-fpz6r 1/1 Running 0 6d19h\n cert-manager-webhook-669b96dcfd-6mnm2 1/1 Running 0 6d19h\n ```\n\n Check `cert-manager` controller logs: \n\n ```\n kubectl -n cert-manager logs cert-manager-66d9545484-772cr | grep \"issuance is skipped until there are no more duplicates\"\n ```\n\n Example outputs: \n\n ```cbm-basic-v2\n 1 controller.go:163] cert-manager/certificates-readiness \"msg\"=\"re-queuing item due to error processing\" \"error\"=\"multiple CertificateRequests were found for the 'next' revision 3, issuance is skipped until there are no more duplicates\" \"key\"=\"apigee-system/apigee-serving-cert\"\n ``` \n\n ```cbm-basic-v2\n 1 controller.go:167] cert-manager/certificates-readiness \"msg\"=\"re-queuing item due to error processing\" \"error\"=\"multiple CertificateRequests were found for the 'next' revision 683, issuance is skipped until there are no more duplicates\" \"key\"=\"apigee/apigee-istiod\"\n ```\n\n\n If you see either of the messages shown above, the `apigee-serving-cert` and the `apigee-istiod-cert` certificates will not be renewed.\n2. List all certificate requests in the `apigee-system` namespace or the `apigee` namespace depending on the namespace printed in the log entries above and check to see if there are multiple certificate requests created for renewing the same `apigee-serving-cert` or `apigee-istiod-cert` certificate revisions: \n\n ```\n kubectl -n apigee-system get certificaterequests\n ```\n\n\nSee the `cert-manager` issue relevant to this problem at\n[cert-manager created multiple CertificateRequest objects with the same\ncertificate-revision](https://github.com/cert-manager/cert-manager/issues/4956).\n\nResolution **Note:** Execute the following commands for the `apigee` or `apigee-system` namespace depending on the namespace where the duplicate certificate requests were found.\n\n1. Delete all certificate requests in `apigee-system` namespace: \n\n ```\n kubectl -n apigee-system delete certificaterequests --all\n ```\n2. Verify that duplicated certificate requests have been deleted and only one certificate request is available for the `apigee-serving-cert` certificate in `apigee-system` namespace: \n\n ```\n kubectl -n apigee-system get certificaterequests\n ```\n3. Verify that the `apigee-serving-cert` certificate has been renewed: \n\n ```\n kubectl -n apigee-system get certificates apigee-serving-cert -o yaml\n ```\n\n An example output: \n\n ```actionscript-3\n apiVersion: cert-manager.io/v1\n kind: Certificate\n metadata:\n creationTimestamp: \"2023-06-26T13:25:10Z\"\n generation: 1\n name: apigee-serving-cert\n namespace: apigee-system\n resourceVersion: \"11053\"\n uid: e7718341-b3ca-4c93-a6d4-30cf70a33e2b\n spec:\n dnsNames:\n - apigee-webhook-service.apigee-system.svc\n - apigee-webhook-service.apigee-system.svc.cluster.local\n issuerRef:\n kind: Issuer\n name: apigee-selfsigned-issuer\n secretName: webhook-server-cert\n status:\n conditions:\n - lastTransitionTime: \"2023-06-26T13:25:11Z\"\n message: Certificate is up to date and has not expired\n observedGeneration: 1\n reason: Ready\n status: \"True\"\n type: Ready\n notAfter: \"2023-09-24T13:25:11Z\"\n notBefore: \"2023-06-26T13:25:11Z\"\n renewalTime: \"2023-08-25T13:25:11Z\"\n revision: 1\n ```\n\nCause: cert-manager is not healthy\n\nDiagnosis\n\n1. Check the health of the `cert-manager` pods in the `cert-manager` namespace: \n\n ```\n kubectl -n cert-manager get pods\n ```\n\n\n If `cert-manager` pods are healthy, all\n `cert-manager` pods should be ready `(1/1)` and in\n `Running` state, otherwise, that could be the reason for this\n issue: \n\n ```text\n NAME READY STATUS RESTARTS AGE\n cert-manager-59cf78f685-mlkvx 1/1 Running 0 15d\n cert-manager-cainjector-78cc865768-krjcp 1/1 Running 0 15d\n cert-manager-webhook-77c4fb46b6-7g9g6 1/1 Running 0 15d\n ```\n2.\n The `cert-manager` can fail for many reasons. Check the\n `cert-manager` logs and identify the reason for the failure\n and resolve them accordingly.\n\n\n One known reason is that the `cert-manager` will fail if it\n cannot communicate with the Kubernetes API. In this case, an error\n message similar to following is displayed:: \n\n ```transact-sql\n E0601 00:10:27.841516 1 leaderelection.go:330] error retrieving\n resource lock kube-system/cert-manager-controller: Get\n \"https://192.168.0.1:443/api/v1/namespaces/kube-system/configmaps/cert-manager-controller\":\n dial tcp 192.168.0.1:443: i/o timeout\n ```\n\nResolution\n\n1. Check the health of the Kubernetes cluster and fix any issues found. See [Troubleshooting Clusters](https://kubernetes.io/docs/tasks/debug/debug-cluster/).\n2. Refer to [Troubleshooting](https://cert-manager.io/docs/troubleshooting) for additional `cert-manager` troubleshooting information.\n\nMust gather diagnostic information\n\n\nIf the problem persists even after following the above instructions, gather\nthe following diagnostic information, and then contact\n[Google Cloud Customer Care](https://cloud.google.com/support-hub/).\n\n1. Google Cloud Project ID\n2. Apigee hybrid organization\n3. Apigee hybrid `overrides.yaml` file, masking any sensitive information.\n4. Kubernetes pod status in all namespaces: \n\n ```\n kubectl get pods -A \u003e kubectl-pod-status`date +%Y.%m.%d_%H.%M.%S`.txt\n ```\n5. Kubernetes `cluster-info` dump: \n\n ```\n # generate kubernetes cluster-info dump\n kubectl cluster-info dump -A --output-directory=/tmp/kubectl-cluster-info-dump\n # zip kubernetes cluster-info dump\n zip -r kubectl-cluster-info-dump`date +%Y.%m.%d_%H.%M.%S`.zip /tmp/kubectl-cluster-info-dump/*\n ```"]]
