Cloud Storage FUSE configuration file

This page describes how to use a Cloud Storage FUSE configuration file to configure the behavior of Cloud Storage FUSE in a persistent manner. To use the configuration file, specify the path to the configuration file in the --config-file flag as part of your mount command.

The configuration file is a YAML file that uses the following format and fields. Some of the fields can also be specified by using command-line options.

app-name: "APP_NAME"
logging:
  file-path: "FILE_PATH"
  format: FORMAT
  severity: SEVERITY
  log-rotate:
    max-file-size-mb: MAX_FILE_SIZE
    backup-file-count: BACKUP_FILE_COUNT
    compress: COMPRESS
file-cache:
  max-size-mb: MAX_SIZE
  cache-file-for-range-read: CACHE_FILE_FOR_RANGE_READ
  enable-parallel-downloads: ENABLE_PARALLEL_DOWNLOADS
  parallel-downloads-per-file: PARALLEL_DOWNLOADS_PER_FILE
  max-parallel-downloads: MAX_PARALLEL_DOWNLOADS
  download-chunk-size-mb: DOWNLOAD_CHUNK_SIZE
metadata-cache:
  enable-nonexistent-type-cache: ENABLE_NONEXISTENT_TYPE_CACHE
  negative-ttl-secs: ENABLE_NEGATIVE_TTL_SECS
  stat-cache-max-size-mb: STAT_CACHE_MAX_SIZE
  ttl-secs: TTL_SECS
  type-cache-max-size-mb: TYPE_CACHE_MAX_SIZE
cache-dir: "CACHE_DIR"
only-dir: "ONLY_DIR"
gcs-auth:
  anonymous-access: ANONYMOUS_ACCESS
  key-file: "KEY_FILE"
  reuse-token-from-url: REUSE_TOKEN_FROM_URL
  token-url: "TOKEN_URL"
gcs-connection:
  billing-project: "BILLING_PROJECT"
  client-protocol: CLIENT_PROTOCOL
  custom-endpoint: "CUSTOM_ENDPOINT"
  http-client-timeout: HTTP_CLIENT_TIMEOUT
  limit-bytes-per-sec: "LIMIT_BYTES_PER_SEC"
  limit-ops-per-sec: "LIMIT_OPS_PER_SEC"
  max-conns-per-host: MAX_CONNS_PER_HOST
  max-idle-conns-per-host: MAX_IDLE_CONNS_PER_HOST
  sequential-read-size-mb: SEQUENTIAL_READ_SIZE
implicit-dirs: IMPLICIT_DIRS
file-system:
  kernel-list-cache-ttl-secs: KERNEL_LIST_CACHE_TTL_SECS
  ignore-interrupts: IGNORE_INTERRUPTS
  dir-mode: "DIR_MODE"
  file-mode: "FILE_MODE"
  fuse-options: FUSE_OPTIONS
  gid: GID
  rename-dir-limit: RENAME_DIR_LIMIT
  temp-dir: "TEMP_DIR"
  uid: UID
foreground: FOREGROUND
gcs-retries:
  max-retry-sleep: MAX_RETRY_SLEEP
  multiplier: "MULTIPLIER"
metrics:
  stackdriver-export-interval: STACKDRIVER_EXPORT_INTERVAL
debug:
  log-mutex: LOG_MUTEX
  exit-on-invariant-violation: EXIT_ON_INVARIANT_VIOLATION
write:
  enable-streaming-writes: STREAMING_WRITES

Configuration fields

The following table describes the fields you can specify in your configuration file. Unless stated otherwise, all fields are optional.

Field Description Valid value Default value
app-name The application name of the mount. String value, for example: "my-bucket-mount". ""
file-path The path to the log file where logs will be written. If this field is unspecified, the logs are routed to stdout when Cloud Storage FUSE runs in foreground mode and to syslogs when Cloud Storage FUSE runs in background mode. String value, for example: "/var/log". ""
format Specifies the format of the log file.
  • text
  • json
 json
severity

The severity level you want Cloud Storage FUSE to generate logs for. The severity levels are ordered from lowest severity to highest severity. For example, when you specify warning, Cloud Storage FUSE generates logs for warnings and errors. Generally, we recommend using the info severity level.
  • trace
  • debug
  • info
  • warning
  • error
  • off: Disables all logging.
 info
max-file-size-mb The maximum size in megabytes (MB) that log files can reach before being rotated. Integer. The minimum value is 1. 512
backup-file-count The maximum number of rotated log files to retain, excluding the active file that logs are written to.
  • Integer
  • 0: Retains all rotated log files
 10
compress Specifies whether rotated log files are compressed using gzip. Boolean value: true, false true
max-size-mb

The maximum size in MiB that the file cache can use. If present, max-size-mb enables file caching in Cloud Storage FUSE and is useful if you want to limit the total capacity the Cloud Storage FUSE cache can use within its mounted directory.
  • Integer
  • -1: Specifies the use of the cache's entire available capacity in the directory you specify for cache-dir.
  • 0: Disables the file cache.
 -1
cache-file-for-range-read Determines whether the full object should be downloaded asynchronously and stored in the Cloud Storage FUSE cache directory when the first read is completed from a non-zero offset. This option should be set to true if you plan to perform several random reads or partial reads.

Note: If you perform a partial read starting at offset 0, Cloud Storage FUSE asynchronously downloads and caches the full object.

 Boolean value: true, false false
enable-parallel-downloads

Accelerates reads of large files by using the file cache directory as a prefetch buffer using multiple workers to download large files in parallel. To learn more about parallel downloads and configure its supporting properties, see Improve read performance using parallel downloads.

To use parallel downloads, you must first enable file caching.

 Boolean value: true, false false
parallel-downloads-per-file

Specifies the number of maximum goroutines to spawn per file to download the object from Cloud Storage into the file cache.

 Integer 16
max-parallel-downloads The maximum number of goroutines that can be spawned at any given time across all the download jobs of files.
  • Integer
  • -1: Specifies unlimited parallel downloads.
  • 0: Disables parallel downloads. Can only be used if --enable-parallel-downloads is not passed or is passed as false.
  • >0: Specifies no upper limit. Cloud Storage FUSE internally limits the value based on the maximum number of goroutines that can be spawned specified by your machine's configuration.
 Twice the number of CPU cores on your machine or 16, whichever is higher.
download-chunk-size-mb Specifies the size of each read request in MiB that each goroutine makes to Cloud Storage when downloading the object into the file cache. Integer 50
enable-nonexistent-type-cache Creates a type cache entry with the type NonexistentType if a file isn't found in Cloud Storage. If the file gets created in Cloud Storage but the NonexistentType entry for the file is cached, then Cloud Storage FUSE cannot request that file until the NonexistentType entry is removed from the type cache. Boolean value: true, false false
stat-cache-max-size-mb The maximum size memory that the stat cache can use, in MiB. The stat cache is always entirely kept in memory.
  • Integer. We recommend the following:
    • 32 if your workload involves up to 20,000 files.
    • If your workload is larger than 20,000 files, increase the size by values of 10 for every additional 6,000 files, where the stat cache uses an average of 1,500 MiB per file.
  • -1: Sets no limit, where the stat cache use as much memory as needed.
  • 0: Disables the stat cache.
 32
negative-ttl-secs

Defines the time to live (TTL) in seconds of negative stat cache entries, which store results for non-existent files in the cache.
  • Integer representing seconds, for example: 10 (10 seconds).
  • 0: Disables negative stat caching.
  • -1: Allows unlimited negative stat caching and disables a TTL expiration.
 5
ttl-secs Defines the time to live (TTL) in seconds of cached metadata entries.
  • Integer representing seconds, for example: 30 (30 seconds).
  • -1: Bypass TTL expiration and serve files from the cache whenever they're available.
  • 0: Use the most up-to-date file. Using this value issues a Get metadata call to make sure that the object generation for the file in the cache matches what's stored in Cloud Storage. To learn more, see Configuring cache invalidation.
 60
type-cache-max-size-mb The maximum size in MiB per directory that the type cache can use. The type cache is always entirely kept in memory.
  • Integer. We recommend the following:
    • 4 if the maximum number of files within a single directory from the bucket you're mounting contains 20,000 files or less.
    • If the maximum number of files within a single directory that you're mounting contains more than 20,000 files, increase the value by 1 for every 5,000 files, which is an average of around 200 bytes per file.
  • -1: Specifies no limit and lets the type cache use as much memory as needed.
  • 0: Disables the type cache.
 4
cache-dir

Specifies the directory for storing file cache data.

To learn how to enable file caching, see Use file caching.

 A path, for example: "/tmp/gcsfuse-cache-path".
only-dir Mounts only a specific directory within a bucket. A path, for example: "/etc/gcsfuse.yaml".
anonymous-access Disables authentication for requests. This option should be set if you're using a custom endpoint that doesn't support authentication. This option should also be set if you're using Cloud Storage FUSE with public buckets. Boolean value: true, false false
key-file Specifies an absolute path to the credential JSON key file for authenticating requests to Cloud Storage. By default, Cloud Storage FUSE uses Application Default Credentials to authenticate requests. A URL. When this option is not set, Application Default Credentials are used.
reuse-token-from-url Specifies whether to reuse the token acquired from --token-url. Boolean value: true, false true
token-url Specifies a URL for getting an access token when the --key-file is absent. A URL.
billing-project Specifies a project to use for billing when the mounted bucket is accessed. This option is often required when mounting a bucket enabled with Requester Pays. String value representing a "project ID". ""
client-protocol Specifies the protocol used for communicating with the Cloud Storage backend.
  • http1 for HTTP/1.1
  • http2 for HTTP/2
  • grpc for gRPC
 http1
custom-endpoint Specifies an alternative custom endpoint for fetching data. The custom endpoint must support the equivalent resources and operations as the Cloud Storage JSON endpoint, https://storage.googleapis.com/storage/v1. If a custom endpoint isn't specified, Cloud Storage FUSE uses the global Cloud Storage JSON API endpoint, https://storage.googleapis.com/storage/v1. If authentication isn't supported on the custom endpoint you specify, set the --anonymous-access flag to true to bypass authentication. An endpoint, for example: "http://localhost:443/storage/v1".
http-client-timeout Specifies how long the Cloud Storage FUSE HTTP client can wait to get a response from the server before timing out. Duration, for example: 1h10m10s for 1 hour, 10 minutes, and 10 seconds. 0s specifies no timeout. 0s, which specifies no timeout
limit-bytes-per-sec Specifies the bandwidth limit at which Cloud Storage FUSE can read data from Cloud Storage, measured over a 30-second window. "-1", which specifies no limit.
limit-ops-per-sec Specifies a limit for operations performed per second, measured over a 30-second window. Floating point number. -1 specifies no limit. "-1"
max-conns-per-host Specifies the maximum number of TCP connections allowed per server. This becomes effective when --client-protocol is set to http1. 0
max-idle-conns-per-host Specifies the maximum number of idle connections allowed per server. This becomes effective when --client-protocol is set to http1. Integer between 0 and 2147483647. 0 specifies no limit on TCP connections. 0
sequential-read-size-mb Specifies the chunk size of the data to be downloaded from Cloud Storage, in megabytes (MB). Integer between 1 and 1024. 200
implicit-dirs Implicitly includes folders and managed folders. See the files and directories documentation in GitHub for more information. Boolean value: true, false false
kernel-list-cache-ttl-secs Enables the list cache and defines the time to live (TTL) in seconds of cached list entries. The list cache is kept in memory in the page cache, which is controlled by the kernel based on available memory.
  • Integer representing seconds, for example: 10 (10 seconds).
  • 0: Disables list caching.
  • -1: Bypasses entry expiration and always returns the list response from the cache when it's available.
0
ignore-interrupts Instructs Cloud Storage FUSE to ignore system interrupt signals, like SIGINT triggered by Control+C. This prevents signals from terminating in-flight operations. Boolean value: true, false. true
dir-mode Permissions bits for directories, in octal. Integer between 000 and 777 (inclusive). "755"
file-mode Specifies permissions bits for files, in octal. Integer between 000 and 777 (inclusive). "644"
fuse-options Specifies additional system-specific mount options.
gid Specifies the Group Identifier (GID) owner of all inodes.
  • Integer representing a GID.
  • -1: The GID of the caller is used.
 -1
rename-dir-limit Allows the renaming of directories containing fewer descendants than the specified limit. Integer between 0 and 2147483647. 0
temp-dir Specifies a path to the temporary directory where writes are staged prior to being uploaded to Cloud Storage. A string path, for example: "/mnt/ssd/example-user-gcsfuse-temp-dir". "/tmp"
uid Specifies the User Identifier (UID) owner of all inodes.
  • Integer representing a UID.
  • -1: The UID of the caller is used.
 -1
foreground Runs the gcsfuse command in the foreground. Boolean value: true, false false
max-retry-sleep Specifies the maximum duration that Cloud Storage FUSE is allowed to sleep in a retry loop with exponential backoff. Once the backoff duration exceeds the specified maximum duration, the retry continues with the specified maximum duration. Duration, for example: 1h5m50s (1 hour, 5 minutes, and 50 seconds) or 60s (60 seconds). 30s
multiplier Specifies the multiplier for exponential backoff between consecutive retries. Floating-point number "2"
stackdriver-export-interval Exports metrics to stackdriver with the specified interval. Duration, for example: 1h5m50s (1 hour, 5 minutes, and 50 seconds). 0s specifies no exporting. 0s
log-mutex Prints debug messages when a mutex is held too long. If this option is specified, the severity level of logs is automatically set to trace, which includes trace logs, debug logs, info logs, warning logs, and error logs. Boolean value: true, false. false
exit-on-invariant-violation Exits the program when internal variant violations are detected. Boolean value: true, false. false
enable-streaming-writes Controls the write path flow so that data is uploaded directly to Cloud Storage as its written instead of fully staging the write locally and uploading it upon close() or fsync(). For more information about streaming writes, see the Cloud Storage FUSE GitHub documentation. Boolean value: true, false. false