Stay organized with collections
Save and categorize content based on your preferences.
Shim for Running gcloud storage
Overview
Cloud SDK includes a new CLI, gcloud storage, that can be considerably faster
than gsutil when performing uploads and downloads with less parameter
tweaking. This new CLI has a syntax and command structure that is familiar to
gsutil users but is fundamentally different in many important ways. To ease
transition to this new CLI, gsutil provides a shim that translates your gsutil
commands to gcloud storage commands if an equivalent exists, and falls back to
gsutil's usual behavior if an equivalent does not exist.
To Enable
Set use_gcloud_storage=True in the .boto config file under the
[GSUtil] section:
[GSUtil]
use_gcloud_storage=True
You can also set the flag for individual commands using the top-level -o
flag:
The gcloud storage CLI only supports a subset of gsutil commands. What follows
is a list of commands supported by the shim with any differences in behavior
noted.
acl
The ch subcommand is not supported.
autoclass
Works as expected.
bucketpolicyonly
Works as expected.
cat
Prints object data for a second object even if the first object is invalid.
compose
Works as expected.
cors
get subcommand prints "[]" instead of "gs://[bucket name] has no CORS
configuration".
cp
Copies a second object even if the first object is invalid.
Does not support file to file copies.
Supports copying objects cloud-to-cloud with trailing slashes in the name.
The all-version flag (-A) silently enables sequential execution rather
than raising an error.
defacl
The ch subcommand is not supported.
defstorageclass
Works as expected.
hash
In gsutil, the -m and -c flags that affect which hashes are displayed
are ignored for cloud objects. This behavior is fixed for the shim and gcloud
storage.
iam
The ch subcommand is not supported.
The -f flag will continue on any error, not just API errors.
kms
The authorize subcommand returns informational messages in a different
format.
The encryption subcommand returns informational messages in a different
format.
labels
get subcommand prints "[]" instead of "gs://[bucket name] has no labels
configuration."
lifecycle
Works as expected.
logging
The get subcommand has different JSON spacing and doesn't print an
informational message if no configuration is found.
ls
Works as expected.
mb
Works as expected.
mv
See notes on cp.
notification
The list subcommand prints configuration information as YAML.
The delete subcommand offers progress tracking and parallelization.
pap
Works as expected.
rb
Works as expected.
requesterpays
Works as expected.
rewrite
The -k flag does not throw an error if called without a new key. In both the
shim and unshimmed cases, the old key is maintained.
rm
$folder$ delete markers are not supported.
rpo
Works as expected.
setmeta
Does not throw an error if no headers are changed.
stat
Includes a field "Storage class update time:" which may throw off tabbing.
ubla
Works as expected.
versioning
Works as expected.
web
The get subcommand has different JSON spacing and doesn't print an
informational message if no configuration is found.
Boto Configuration
Configuration found in the boto file is mapped 1:1 to gcloud environment
variables where appropriate.
Due to its compatibility across all major platforms, multiprocessing is
enabled for all commands by default (equivalent to the -m option always
being included in gsutil).
A sequence of asterisks greater than 2 (i.e. ***) are always treated as
a single asterisk.
Unlike gsutil, gcloud is not designed to be used in parallel invocations,
and doing so (i.e. running the shim from 2 terminals at once) can lead to
unpredictable behavior.
Assuming a bucket contains an object gs://bucket/nested/foo.txt,
gsutil's wildcard iterator will match foo.txt given a URL like
gs://bucket/*/nested/*. The shim will not match foo.txt given the
same URL.
This will be updated as new commands are supported by both gcloud storage
and the shim.
If Unicode is having issues, try setting the environment variable
PYTHONUTF8 to 1. Specifically, this may help on the Windows
command-line (CMD).
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-28 UTC."],[],[],null,["# Shim for Running gcloud storage\n===============================\n\n\u003cbr /\u003e\n\n| **Important:** gsutil is not the recommended CLI for Cloud Storage. [Use\n| `gcloud storage`](/storage/docs/discover-object-storage-gcloud) commands in the Google Cloud CLI instead.\n\n\u003cbr /\u003e\n\nOverview\n--------\n\nCloud SDK includes a new CLI, gcloud storage, that can be considerably faster\nthan gsutil when performing uploads and downloads with less parameter\ntweaking. This new CLI has a syntax and command structure that is familiar to\ngsutil users but is fundamentally different in many important ways. To ease\ntransition to this new CLI, gsutil provides a shim that translates your gsutil\ncommands to gcloud storage commands if an equivalent exists, and falls back to\ngsutil's usual behavior if an equivalent does not exist.\n\nTo Enable\n---------\n\nSet `use_gcloud_storage=True` in the `.boto` config file under the\n`[GSUtil]` section: \n\n```\n[GSUtil]\nuse_gcloud_storage=True\n```\n\nYou can also set the flag for individual commands using the top-level `-o`\nflag: \n\n```\ngsutil -o \"GSUtil:use_gcloud_storage=True\" -m cp -p file gs://bucket/obj\n```\n\nAvailable Commands\n------------------\n\nThe gcloud storage CLI only supports a subset of gsutil commands. What follows\nis a list of commands supported by the shim with any differences in behavior\nnoted.\n\n### acl\n\n- The `ch` subcommand is not supported.\n\n### autoclass\n\n- Works as expected.\n\n### bucketpolicyonly\n\n- Works as expected.\n\n### cat\n\n- Prints object data for a second object even if the first object is invalid.\n\n### compose\n\n- Works as expected.\n\n### cors\n\n- `get` subcommand prints \"\\[\\]\" instead of \"gs://\\[bucket name\\] has no CORS\n configuration\".\n\n### cp\n\n- Copies a second object even if the first object is invalid.\n\n- Does not support file to file copies.\n\n- Supports copying objects cloud-to-cloud with trailing slashes in the name.\n\n- The all-version flag (`-A`) silently enables sequential execution rather\n than raising an error.\n\n### defacl\n\n- The `ch` subcommand is not supported.\n\n### defstorageclass\n\n- Works as expected.\n\n### hash\n\n- In gsutil, the `-m` and `-c` flags that affect which hashes are displayed\n are ignored for cloud objects. This behavior is fixed for the shim and gcloud\n storage.\n\n### iam\n\n- The `ch` subcommand is not supported.\n\n- The `-f` flag will continue on any error, not just API errors.\n\n### kms\n\n- The authorize subcommand returns informational messages in a different\n format.\n\n- The encryption subcommand returns informational messages in a different\n format.\n\n### labels\n\n- `get` subcommand prints \"\\[\\]\" instead of \"gs://\\[bucket name\\] has no labels\n configuration.\"\n\n### lifecycle\n\n- Works as expected.\n\n### logging\n\n- The get subcommand has different JSON spacing and doesn't print an\n informational message if no configuration is found.\n\n### ls\n\n- Works as expected.\n\n### mb\n\n- Works as expected.\n\n### mv\n\n- See notes on cp.\n\n### notification\n\n- The list subcommand prints configuration information as YAML.\n\n- The delete subcommand offers progress tracking and parallelization.\n\n### pap\n\n- Works as expected.\n\n### rb\n\n- Works as expected.\n\n### requesterpays\n\n- Works as expected.\n\n### rewrite\n\n- The -k flag does not throw an error if called without a new key. In both the\n shim and unshimmed cases, the old key is maintained.\n\n### rm\n\n- `$folder$` delete markers are not supported.\n\n### rpo\n\n- Works as expected.\n\n### setmeta\n\n- Does not throw an error if no headers are changed.\n\n### stat\n\n- Includes a field \"Storage class update time:\" which may throw off tabbing.\n\n### ubla\n\n- Works as expected.\n\n### versioning\n\n- Works as expected.\n\n### web\n\n- The get subcommand has different JSON spacing and doesn't print an\n informational message if no configuration is found.\n\nBoto Configuration\n------------------\n\nConfiguration found in the boto file is mapped 1:1 to gcloud environment\nvariables where appropriate.\n\n### \\[Credentials\\]\n\n- aws_access_key_id: AWS_ACCESS_KEY_ID\n\n- aws_secret_access_key: AWS_SECRET_ACCESS_KEY\n\n- use_client_certificate: CLOUDSDK_CONTEXT_AWARE_USE_CLIENT_CERTIFICATE\n\n### \\[Boto\\]\n\n- proxy: CLOUDSDK_PROXY_ADDRESS\n\n- proxy_type: CLOUDSDK_PROXY_TYPE\n\n- proxy_port: CLOUDSDK_PROXY_PORT\n\n- proxy_user: CLOUDSDK_PROXY_USERNAME\n\n- proxy_pass: CLOUDSDK_PROXY_PASSWORD\n\n- proxy_rdns: CLOUDSDK_PROXY_RDNS\n\n- http_socket_timeout: CLOUDSDK_CORE_HTTP_TIMEOUT\n\n- ca_certificates_file: CLOUDSDK_CORE_CUSTOM_CA_CERTS_FILE\n\n- max_retry_delay: CLOUDSDK_STORAGE_BASE_RETRY_DELAY\n\n- num_retries: CLOUDSDK_STORAGE_MAX_RETRIES\n\n### \\[GSUtil\\]\n\n- check_hashes: CLOUDSDK_STORAGE_CHECK_HASHES\n\n- default_project_id: CLOUDSDK_CORE_PROJECT\n\n- disable_analytics_prompt: CLOUDSDK_CORE_DISABLE_USAGE_REPORTING\n\n- use_magicfile: CLOUDSDK_STORAGE_USE_MAGICFILE\n\n- parallel_composite_upload_threshold: CLOUDSDK_STORAGE_PARALLEL_COMPOSITE_UPLOAD_THRESHOLD\n\n- resumable_threshold: CLOUDSDK_STORAGE_RESUMABLE_THRESHOLD\n\n### \\[OAuth2\\]\n\n- client_id: CLOUDSDK_AUTH_CLIENT_ID\n\n- client_secret: CLOUDSDK_AUTH_CLIENT_SECRET\n\n- provider_authorization_uri: CLOUDSDK_AUTH_AUTH_HOST\n\n- provider_token_uri: CLOUDSDK_AUTH_TOKEN_HOST\n\nGeneral Compatibility Notes\n---------------------------\n\n- Due to its compatibility across all major platforms, multiprocessing is\n enabled for all commands by default (equivalent to the -m option always\n being included in gsutil).\n\n- A sequence of asterisks greater than 2 (i.e. `***`) are always treated as\n a single asterisk.\n\n- Unlike gsutil, gcloud is not designed to be used in parallel invocations,\n and doing so (i.e. running the shim from 2 terminals at once) can lead to\n unpredictable behavior.\n\n- Assuming a bucket contains an object `gs://bucket/nested/foo.txt`,\n gsutil's wildcard iterator will match `foo.txt` given a URL like\n `gs://bucket/*/nested/*`. The shim will not match `foo.txt` given the\n same URL.\n\n- This will be updated as new commands are supported by both gcloud storage\n and the shim.\n\n- If Unicode is having issues, try setting the environment variable\n `PYTHONUTF8` to `1`. Specifically, this may help on the Windows\n command-line (CMD)."]]
