When building, testing, and running a workload, it can be useful to monitor its
progress to debug issues. The following tools are available to use for
monitoring and debugging:
The debug Confidential Space image: The
debug Confidential Space image
keeps the Confidential VM running the workload operational after the workload has
completed, and runs an SSH server. This lets you remotely log into the VM to
diagnose issues. It's useful to use the debug image until you're confident
that your code is behaving as it should. When it's time to start working on
sensitive production data, switch to the production Confidential Space
image.
Interactive shell: After using SSH to connect to your workload
Confidential VM, you can use the
sudo ctr task exec -t --exec-id shell tee-container bash command to enter
an interactive shell inside the container to diagnose workload issues.
Logging
Like any command line program, the workload STDOUT and STDERR can be
displayed in the console. It can also be redirected to Cloud Logging by the
workload operator setting the
tee-container-log-redirect
metadata key to true or cloud_logging on the Confidential Space VM, and
ensuring that the service account running the workload has the
logging.logWriter role.
To reduce your risk profile, log the minimum amount of information, and don't
log sensitive information.
View Confidential Space logs
If the service account attached to your Confidential Space VM has been granted the
logging.logWriter role and you've redirected logs to Cloud Logging, you can
troubleshoot errors by viewing the VM's logs:
Go to Logging in the workload operator's project in the
Google Cloud console.
Next to the Query tab, click the time range to set the logging period
you want to view.
Filter the logs by the following log fields if they're available:
Resource type: VM Instance
Instance ID: The instance ID of the Confidential VM
Log name: confidential-space-launcher
Read the failure message to find out what the problem is. A resource might
not have been set up properly, the attribute conditions in your data
collaborators' WIP providers might not match the claims made by the
Confidential Space workload, or the workload itself might have had an error.
Return codes
Return codes are displayed in the console when running the
launcher
and workload, and can be redirected to Cloud Logging.
The return codes are described in the following table:
Code
Definition
VM stop behavior
0
The workload completed successfully
when using the production image.
The VM stops after the
workload is complete.
1
The workload or launcher returned
an error when using the production
image.
The VM stops after it has
returned an error.
3
The launcher has restarted after a
failure due to its
tee-restart-policy.
The VM is restarted.
4
The workload or launcher has
finished running when using the
debug image, and the VM is now
idling.
The VM doesn't stop after the
workload completes or returns
an error. This is so you can
debug the workload over SSH.
If a workload fails, a workload operator only receives the message
workload finished with a non-zero return code, without further context. For a
production image, the launcher can be set to restart on failure with
tee-restart-policy=OnFailure.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-29 UTC."],[[["\u003cp\u003eCloud Logging can be used to troubleshoot Confidential Space workloads by redirecting \u003ccode\u003eSTDOUT\u003c/code\u003e and \u003ccode\u003eSTDERR\u003c/code\u003e and checking for workload return codes to identify failure points.\u003c/p\u003e\n"],["\u003cp\u003eThe debug Confidential Space image allows for remote SSH access to the VM after workload completion, enabling in-depth diagnosis of issues before switching to the production image.\u003c/p\u003e\n"],["\u003cp\u003eMemory usage monitoring can be enabled for workloads, providing visibility through Cloud Logging or Metrics Explorer, but it requires enablement by both the workload author and operator.\u003c/p\u003e\n"],["\u003cp\u003eWorkload operators can use an interactive shell within the container of a Confidential VM to diagnose issues after connecting via SSH.\u003c/p\u003e\n"],["\u003cp\u003eReturn codes from the workload or launcher, which are displayed in the console and can be redirected to cloud logging, provide crucial information about the success or failure of the workload.\u003c/p\u003e\n"]]],[],null,["# Monitor and debug workloads\n\n[Workload author](/confidential-computing/confidential-space/docs/confidential-space-overview#roles) [Workload operator](/confidential-computing/confidential-space/docs/confidential-space-overview#roles)\n\n*** ** * ** ***\n\nWhen building, testing, and running a workload, it can be useful to monitor its\nprogress to debug issues. The following tools are available to use for\nmonitoring and debugging:\n\n- **Cloud Logging** : As the first step in troubleshooting a Confidential Space\n workload, you can\n [redirect `STDOUT` and `STDERR` to Cloud Logging](/confidential-computing/confidential-space/docs/deploy-workloads#tee-container-log-redirect),\n and then\n [check it for workload return codes](#logging) to see where a failure\n occurred.\n\n- **The debug Confidential Space image** : The\n [debug Confidential Space image](/confidential-computing/confidential-space/docs/confidential-space-images#types_of_images)\n keeps the Confidential VM running the workload operational after the workload has\n completed, and runs an SSH server. This lets you remotely log into the VM to\n diagnose issues. It's useful to use the debug image until you're confident\n that your code is behaving as it should. When it's time to start working on\n sensitive production data, switch to the production Confidential Space\n image.\n\n- **Memory usage monitoring** : You can view the memory usage of the workload in\n [Cloud Logging](/logging) or\n [Metrics Explorer](/monitoring/charts/metrics-explorer).\n The\n [workload author needs to allow it](/confidential-computing/confidential-space/docs/reference/launch-policies#monitoring-memory-allow),\n and the\n [workload operator needs to enable it](/confidential-computing/confidential-space/docs/reference/metadata-variables#tee-memory-monitoring-enable)\n before memory usage is tracked.\n\n- **Interactive shell** : After using SSH to connect to your workload\n Confidential VM, you can use the\n `sudo ctr task exec -t --exec-id shell tee-container bash` command to enter\n an interactive shell inside the container to diagnose workload issues.\n\nLogging\n-------\n\nLike any command line program, the workload `STDOUT` and `STDERR` can be\ndisplayed in the console. It can also be redirected to Cloud Logging by the\nworkload operator setting the\n[`tee-container-log-redirect`](/confidential-computing/confidential-space/docs/deploy-workloads#tee-container-log-redirect)\nmetadata key to `true` or `cloud_logging` on the Confidential Space VM, and\nensuring that the service account running the workload has the\n`logging.logWriter` role.\n\nRedirection can be prevented by the workload author with the\n[`log_redirect` launch policy](/confidential-computing/confidential-space/docs/create-customize-workloads#log-redirect).\n\nTo reduce your risk profile, log the minimum amount of information, and don't\nlog sensitive information.\n\n### View Confidential Space logs\n\nIf the service account attached to your Confidential Space VM has been granted the\n`logging.logWriter` role and you've redirected logs to Cloud Logging, you can\ntroubleshoot errors by viewing the VM's logs:\n\n1. Go to **Logging** in the workload operator's project in the\n Google Cloud console.\n\n [Go to Logging](https://console.cloud.google.com/logs)\n2. Next to the **Query** tab, click the time range to set the logging period\n you want to view.\n\n3. Filter the logs by the following log fields if they're available:\n\n - **Resource type:** VM Instance\n\n - **Instance ID:** The instance ID of the Confidential VM\n\n - **Log name:** confidential-space-launcher\n\n4. Read the failure message to find out what the problem is. A resource might\n not have been set up properly, the attribute conditions in your data\n collaborators' WIP providers might not match the claims made by the\n Confidential Space workload, or the workload itself might have had an error.\n\nReturn codes\n------------\n\nReturn codes are displayed in the console when running the\n[launcher](/docs/security/confidential-space#attestation-process)\nand workload, and can be redirected to Cloud Logging.\n\nThe return codes are described in the following table:\n\nIf a workload fails, a workload operator only receives the message\n`workload finished with a non-zero return code`, without further context. For a\nproduction image, the launcher can be set to restart on failure with\n[`tee-restart-policy=OnFailure`](/confidential-computing/confidential-space/docs/deploy-workloads#tee-restart-policy)."]]
