Deployment proxy API gagal dengan apigee-serving-cert tidak ditemukan atau habis masa berlakunya
Tetap teratur dengan koleksi
Simpan dan kategorikan konten berdasarkan preferensi Anda.
Anda sedang melihat dokumentasi Apigee dan Apigee hybrid.
Lihat dokumentasi
Apigee Edge.
Gejala
Deployment proxy API gagal dengan pesan error berikut.
Pesan Error
Jika sertifikat TLS layanan
apigee-webhook-service.apigee-system.svc telah habis masa berlakunya
atau belum valid, pesan error berikut akan ditampilkan di
log apigee-watcher:
Jika ada permintaan sertifikat duplikat yang dibuat untuk memperpanjang sertifikat apigee-serving-cert, sertifikat apigee-serving-cert mungkin tidak diperpanjang.
Pastikan untuk menyertakan semua setelan yang ditampilkan, termasuk --atomic
sehingga tindakan akan di-roll back jika gagal.
Pastikan sertifikat apigee-serving-cert telah
dibuat:
kubectl -n apigee-system get certificates apigee-serving-cert
Penyebab: Permintaan sertifikat duplikat dibuat untuk memperpanjang apigee-serving-cert
Diagnosis
Periksa log pengontrol cert-manager dan lihat apakah pesan error
yang mirip dengan yang berikut ini telah ditampilkan.
Mencantumkan semua pod cert-manager:
kubectl -n cert-manager get pods
Contoh output:
NAME READY STATUS RESTARTS AGE
cert-manager-66d9545484-772cr 1/1 Running 0 6d19h
cert-manager-cainjector-7d8b6bd6fb-fpz6r 1/1 Running 0 6d19h
cert-manager-webhook-669b96dcfd-6mnm2 1/1 Running 0 6d19h
Periksa log pengontrol cert-manager:
kubectl -n cert-manager logs cert-manager-66d9545484-772cr | grep "issuance is skipped until there are no more duplicates"
Contoh output:
1controller.go:163]cert-manager/certificates-readiness"msg"="re-queuing item due to error processing""error"="multiple CertificateRequests were found for the 'next' revision 3, issuance is skipped until there are no more duplicates""key"="apigee-system/apigee-serving-cert"
1controller.go:167]cert-manager/certificates-readiness"msg"="re-queuing item due to error processing""error"="multiple CertificateRequests were found for the 'next' revision 683, issuance is skipped until there are no more duplicates""key"="apigee/apigee-istiod"
Jika Anda melihat salah satu pesan yang ditampilkan di atas, sertifikat apigee-serving-cert dan apigee-istiod-cert tidak akan diperpanjang.
Cantumkan semua permintaan sertifikat di namespace apigee-system atau namespace apigee,
bergantung pada namespace yang dicetak dalam entri log di atas, dan periksa untuk melihat apakah ada beberapa permintaan sertifikat yang dibuat untuk
memperpanjang revisi sertifikat apigee-serving-cert atau apigee-istiod-cert yang sama:
Pastikan permintaan sertifikat duplikat telah dihapus dan hanya satu
permintaan sertifikat yang tersedia untuk sertifikat
apigee-serving-cert di namespace apigee-system:
kubectl -n apigee-system get certificaterequests
Pastikan sertifikat apigee-serving-cert telah diperpanjang:
kubectl -n apigee-system get certificates apigee-serving-cert -o yaml
Periksa status pod cert-manager di
namespace cert-manager:
kubectl -n cert-manager get pods
Jika pod cert-manager sehat, semua pod cert-manager harus siap (1/1) dan dalam status Running. Jika tidak, hal ini dapat menjadi alasan masalah ini:
NAME READY STATUS RESTARTS AGE
cert-manager-59cf78f685-mlkvx 1/1 Running 0 15d
cert-manager-cainjector-78cc865768-krjcp 1/1 Running 0 15d
cert-manager-webhook-77c4fb46b6-7g9g6 1/1 Running 0 15d
cert-manager dapat gagal karena banyak alasan. Periksa log cert-manager dan identifikasi alasan kegagalan, lalu selesaikan sesuai kebutuhan.
Salah satu alasan yang diketahui adalah cert-manager akan gagal jika tidak dapat berkomunikasi dengan Kubernetes API. Dalam hal ini, pesan error
yang mirip dengan berikut akan ditampilkan:
Periksa responsivitas cluster Kubernetes dan perbaiki masalah yang ditemukan.
Lihat
Memecahkan Masalah Cluster.
Lihat
Pemecahan masalah untuk mengetahui informasi pemecahan masalah
cert-manager tambahan.
Harus mengumpulkan informasi diagnostik
Jika masalah terus berlanjut meskipun setelah mengikuti petunjuk di atas, kumpulkan informasi diagnostik berikut, lalu hubungi Layanan Pelanggan Google Cloud.
Google Cloud Project ID
Organisasi Apigee Hybrid
File overrides.yaml Apigee hybrid,
yang menyamarkan informasi sensitif apa pun.
Status pod Kubernetes di semua namespace:
kubectl get pods -A > kubectl-pod-status`date +%Y.%m.%d_%H.%M.%S`.txt
Dump cluster-info Kubernetes:
# generate kubernetes cluster-info dump
kubectl cluster-info dump -A --output-directory=/tmp/kubectl-cluster-info-dump
# zip kubernetes cluster-info dump
zip -r kubectl-cluster-info-dump`date +%Y.%m.%d_%H.%M.%S`.zip /tmp/kubectl-cluster-info-dump/*
[[["Mudah dipahami","easyToUnderstand","thumb-up"],["Memecahkan masalah saya","solvedMyProblem","thumb-up"],["Lainnya","otherUp","thumb-up"]],[["Sulit dipahami","hardToUnderstand","thumb-down"],["Informasi atau kode contoh salah","incorrectInformationOrSampleCode","thumb-down"],["Informasi/contoh yang saya butuhkan tidak ada","missingTheInformationSamplesINeed","thumb-down"],["Masalah terjemahan","translationIssue","thumb-down"],["Lainnya","otherDown","thumb-down"]],["Terakhir diperbarui pada 2025-09-05 UTC."],[[["\u003cp\u003eAPI proxy deployments in Apigee hybrid may fail due to an expired or invalid TLS certificate for the \u003ccode\u003eapigee-webhook-service.apigee-system.svc\u003c/code\u003e, leading to errors in \u003ccode\u003eapigee-watcher\u003c/code\u003e logs.\u003c/p\u003e\n"],["\u003cp\u003eThe issue can be caused by the \u003ccode\u003eapigee-serving-cert\u003c/code\u003e not being found, duplicate certificate requests, or an unhealthy \u003ccode\u003ecert-manager\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eResolutions involve updating the \u003ccode\u003eapigee-serving-cert\u003c/code\u003e using Helm, deleting duplicate certificate requests in the \u003ccode\u003eapigee-system\u003c/code\u003e namespace, and ensuring the \u003ccode\u003ecert-manager\u003c/code\u003e is healthy.\u003c/p\u003e\n"],["\u003cp\u003eIf problems persist, diagnostic information such as Google Cloud Project ID, Apigee hybrid organization details, \u003ccode\u003eoverrides.yaml\u003c/code\u003e file, Kubernetes pod status, and a Kubernetes \u003ccode\u003ecluster-info\u003c/code\u003e dump should be gathered for Google Cloud Customer Care.\u003c/p\u003e\n"]]],[],null,["*You're viewing **Apigee** and **Apigee hybrid** documentation.\nView [Apigee Edge](https://docs.apigee.com/api-platform/troubleshoot/support-d) documentation.*\n| **Note:** This document is applicable for Apigee hybrid users only.\n\nSymptoms\n\n\nAPI proxy deployments fail with the following error messages.\n\nError Messages\n\n\nIf the TLS certificate of the\n`apigee-webhook-service.apigee-system.svc` service has expired\nor is not yet valid, the following error message will be shown on\n`apigee-watcher` logs: \n\n```transact-sql\n{\"level\":\"error\",\"ts\":1687991930.7745812,\"caller\":\"watcher/watcher.go:60\",\n\"msg\":\"error during watch\",\"name\":\"ingress\",\"error\":\"INTERNAL: INTERNAL: failed\nto update ApigeeRoute [org-env]-group-84a6bb5, namespace apigee:\nInternal error occurred: failed calling webhook\n\\\"mapigeeroute.apigee.cloud.google.com\\\": Post\n\\\"https://apigee-webhook-service.apigee-system.svc:443/mutate-apigee-cloud-google-com-v1alpha1-apigeeroute?timeout=30s\\\":\nx509:\ncertificate has expired or is not yet valid: current time\n2023-06-28T22:38:50Z is after 2023-06-17T17:14:13Z, INTERNAL: failed to update\nApigeeRoute [org-env]-group-e7b3ff6, namespace apigee \n```\n\nPossible Causes\n\n| Cause | Description |\n|----------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| [The apigee-serving-cert is not found](#not-found) | If the `apigee-serving-cert` is not found in the `apigee-system` namespace, this issue could occur. |\n| [Duplicate certificate requests were created for renewing `apigee-serving-cert`](#duplicate) | If there are duplicate certificate requests created for renewing the `apigee-serving-cert` certificate, the `apigee-serving-cert` certificate may not get renewed. |\n| [cert-manager is not healthy](#not-healthy) | If `cert-manager` is not healthy, the `apigee-serving-cert` certificate may not get renewed. |\n\nCause: The apigee-serving-cert is not found\n\nDiagnosis\n\n1.\n Check the availability of the `apigee-serving-cert` certificate\n in the `apigee-system` namespace:\n\n ```\n kubectl -n apigee-system get certificates apigee-serving-cert\n ```\n\n\n If this certificate is available, an output similar to following should\n be seen: \n\n ```text\n NAME READY SECRET AGE\n apigee-serving-cert True webhook-server-cert 2d10h\n ```\n2. If the apigee-serving-cert certificate is not found in the `apigee-system` namespace, that could be the reason for this issue.\n\nResolution\n\n1. Update the `apigee-serving-cert` using [Helm](/apigee/docs/hybrid/latest/helm-reference): \n\n ```\n helm upgrade ENV_NAME apigee-env/ \\\n --namespace APIGEE_NAMESPACE \\\n --set env=ENV_NAME \\\n --atomic \\\n -f OVERRIDES_FILE\n ```\n\n Make sure to include all of the settings shown, including `--atomic`\n so that the action rolls back on failure.\n2. Verify that the `apigee-serving-cert` certificate has been created: \n\n ```\n kubectl -n apigee-system get certificates apigee-serving-cert\n ```\n\nCause: Duplicate certificate requests were created for\nrenewing apigee-serving-cert\n\nDiagnosis\n\n1.\n Check `cert-manager` controller logs and see whether an error\n message similar to the following has been returned.\n\n List all `cert-manager` pods: \n\n ```\n kubectl -n cert-manager get pods\n ```\n\n An example output: \n\n ```text\n NAME READY STATUS RESTARTS AGE\n cert-manager-66d9545484-772cr 1/1 Running 0 6d19h\n cert-manager-cainjector-7d8b6bd6fb-fpz6r 1/1 Running 0 6d19h\n cert-manager-webhook-669b96dcfd-6mnm2 1/1 Running 0 6d19h\n ```\n\n Check `cert-manager` controller logs: \n\n ```\n kubectl -n cert-manager logs cert-manager-66d9545484-772cr | grep \"issuance is skipped until there are no more duplicates\"\n ```\n\n Example outputs: \n\n ```cbm-basic-v2\n 1 controller.go:163] cert-manager/certificates-readiness \"msg\"=\"re-queuing item due to error processing\" \"error\"=\"multiple CertificateRequests were found for the 'next' revision 3, issuance is skipped until there are no more duplicates\" \"key\"=\"apigee-system/apigee-serving-cert\"\n ``` \n\n ```cbm-basic-v2\n 1 controller.go:167] cert-manager/certificates-readiness \"msg\"=\"re-queuing item due to error processing\" \"error\"=\"multiple CertificateRequests were found for the 'next' revision 683, issuance is skipped until there are no more duplicates\" \"key\"=\"apigee/apigee-istiod\"\n ```\n\n\n If you see either of the messages shown above, the `apigee-serving-cert` and the `apigee-istiod-cert` certificates will not be renewed.\n2. List all certificate requests in the `apigee-system` namespace or the `apigee` namespace depending on the namespace printed in the log entries above and check to see if there are multiple certificate requests created for renewing the same `apigee-serving-cert` or `apigee-istiod-cert` certificate revisions: \n\n ```\n kubectl -n apigee-system get certificaterequests\n ```\n\n\nSee the `cert-manager` issue relevant to this problem at\n[cert-manager created multiple CertificateRequest objects with the same\ncertificate-revision](https://github.com/cert-manager/cert-manager/issues/4956).\n\nResolution **Note:** Execute the following commands for the `apigee` or `apigee-system` namespace depending on the namespace where the duplicate certificate requests were found.\n\n1. Delete all certificate requests in `apigee-system` namespace: \n\n ```\n kubectl -n apigee-system delete certificaterequests --all\n ```\n2. Verify that duplicated certificate requests have been deleted and only one certificate request is available for the `apigee-serving-cert` certificate in `apigee-system` namespace: \n\n ```\n kubectl -n apigee-system get certificaterequests\n ```\n3. Verify that the `apigee-serving-cert` certificate has been renewed: \n\n ```\n kubectl -n apigee-system get certificates apigee-serving-cert -o yaml\n ```\n\n An example output: \n\n ```actionscript-3\n apiVersion: cert-manager.io/v1\n kind: Certificate\n metadata:\n creationTimestamp: \"2023-06-26T13:25:10Z\"\n generation: 1\n name: apigee-serving-cert\n namespace: apigee-system\n resourceVersion: \"11053\"\n uid: e7718341-b3ca-4c93-a6d4-30cf70a33e2b\n spec:\n dnsNames:\n - apigee-webhook-service.apigee-system.svc\n - apigee-webhook-service.apigee-system.svc.cluster.local\n issuerRef:\n kind: Issuer\n name: apigee-selfsigned-issuer\n secretName: webhook-server-cert\n status:\n conditions:\n - lastTransitionTime: \"2023-06-26T13:25:11Z\"\n message: Certificate is up to date and has not expired\n observedGeneration: 1\n reason: Ready\n status: \"True\"\n type: Ready\n notAfter: \"2023-09-24T13:25:11Z\"\n notBefore: \"2023-06-26T13:25:11Z\"\n renewalTime: \"2023-08-25T13:25:11Z\"\n revision: 1\n ```\n\nCause: cert-manager is not healthy\n\nDiagnosis\n\n1. Check the health of the `cert-manager` pods in the `cert-manager` namespace: \n\n ```\n kubectl -n cert-manager get pods\n ```\n\n\n If `cert-manager` pods are healthy, all\n `cert-manager` pods should be ready `(1/1)` and in\n `Running` state, otherwise, that could be the reason for this\n issue: \n\n ```text\n NAME READY STATUS RESTARTS AGE\n cert-manager-59cf78f685-mlkvx 1/1 Running 0 15d\n cert-manager-cainjector-78cc865768-krjcp 1/1 Running 0 15d\n cert-manager-webhook-77c4fb46b6-7g9g6 1/1 Running 0 15d\n ```\n2.\n The `cert-manager` can fail for many reasons. Check the\n `cert-manager` logs and identify the reason for the failure\n and resolve them accordingly.\n\n\n One known reason is that the `cert-manager` will fail if it\n cannot communicate with the Kubernetes API. In this case, an error\n message similar to following is displayed:: \n\n ```transact-sql\n E0601 00:10:27.841516 1 leaderelection.go:330] error retrieving\n resource lock kube-system/cert-manager-controller: Get\n \"https://192.168.0.1:443/api/v1/namespaces/kube-system/configmaps/cert-manager-controller\":\n dial tcp 192.168.0.1:443: i/o timeout\n ```\n\nResolution\n\n1. Check the health of the Kubernetes cluster and fix any issues found. See [Troubleshooting Clusters](https://kubernetes.io/docs/tasks/debug/debug-cluster/).\n2. Refer to [Troubleshooting](https://cert-manager.io/docs/troubleshooting) for additional `cert-manager` troubleshooting information.\n\nMust gather diagnostic information\n\n\nIf the problem persists even after following the above instructions, gather\nthe following diagnostic information, and then contact\n[Google Cloud Customer Care](https://cloud.google.com/support-hub/).\n\n1. Google Cloud Project ID\n2. Apigee hybrid organization\n3. Apigee hybrid `overrides.yaml` file, masking any sensitive information.\n4. Kubernetes pod status in all namespaces: \n\n ```\n kubectl get pods -A \u003e kubectl-pod-status`date +%Y.%m.%d_%H.%M.%S`.txt\n ```\n5. Kubernetes `cluster-info` dump: \n\n ```\n # generate kubernetes cluster-info dump\n kubectl cluster-info dump -A --output-directory=/tmp/kubectl-cluster-info-dump\n # zip kubernetes cluster-info dump\n zip -r kubectl-cluster-info-dump`date +%Y.%m.%d_%H.%M.%S`.zip /tmp/kubectl-cluster-info-dump/*\n ```"]]
