Cloud Storage FUSE configuration file

This page describes how to use a Cloud Storage FUSE configuration file to configure the behavior of Cloud Storage FUSE in a persistent manner. To use the configuration file, specify the path to the configuration file in the --config-file flag as part of your mount command.

The configuration file is a YAML file that uses the following format and fields. Some of the fields can also be specified by using command-line options.

write:
  create-empty-file: CREATE_EMPTY_FILE
logging:
  file-path: "FILE_PATH"
  format: FORMAT
  severity: SEVERITY
  log-rotate:
    max-file-size-mb: MAX_FILE_SIZE
    backup-file-count: BACKUP_FILE_COUNT
    compress: COMPRESS
file-cache:
  max-size-mb: MAX_SIZE
  cache-file-for-range-read: CACHE_FILE_FOR_RANGE_READ
  enable-parallel-downloads: ENABLE_PARALLEL_DOWNLOADS
  parallel-downloads-per-file: PARALLEL_DOWNLOADS_PER_FILE
  max-parallel-downloads: MAX_PARALLEL_DOWNLOADS
  download-chunk-size-mb: DOWNLOAD_CHUNK_SIZE
metadata-cache:
  stat-cache-max-size-mb: STAT_CACHE_MAX_SIZE
  ttl-secs: TTL_SECS
  type-cache-max-size-mb: TYPE_CACHE_MAX_SIZE
cache-dir: "CACHE_DIR"
gcs-auth:
  anonymous-access: ANONYMOUS_ACCESS
file-system:
  kernel-list-cache-ttl-secs: KERNEL_LIST_CACHE_TTL_SECS
  ignore-interrupts: IGNORE_INTERRUPTS

Configuration fields

The following table describes the fields you can specify in your configuration file. Unless stated otherwise, all fields are optional.

Field Description
create-empty-file A Boolean value that specifies whether or not to create a zero-byte file in Cloud Storage when you start writing to a new file in the mounted bucket. If this field is unspecified, the value defaults to false.
file-path The path to the log file where logs will be written, specified as a string. For example, /var/log. If this field is unspecified, the logs are routed to stdout when Cloud Storage FUSE runs in foreground mode and to syslogs when Cloud Storage FUSE runs in background mode.
format The format in which logs are generated, expressed as an enum. Values are text or json. If this field is unspecified, the value defaults to json.
severity

The severity of logs you want Cloud Storage FUSE to generate, expressed as an enum. The severity levels are ordered from lowest severity to highest severity:

  • trace
  • debug
  • info
  • warning
  • error

When you specify a severity level, Cloud Storage FUSE generates logs having equal or higher severity. For example, when you specify warning, Cloud Storage FUSE generates logs for warnings and errors. You can also specify off to turn off all logging. If any of the following three options are also specified as part of your command, the severity level is automatically set to trace:

  • --debug_fuse
  • --debug_gcs
  • --debug_mutex

If this field is unspecified, the value defaults to info.
max-file-size-mb The maximum size in megabytes (MB) that log files can reach before being rotated. The minimum value is 1. If this field is unspecified, the value defaults to 512.
backup-file-count The maximum number of rotated log files to retain, excluding the active file to which logs are written. When the value is set to 0, all rotated log files are retained. If this field is unspecified, the value defaults to 10.
compress A Boolean value that specifies whether rotated log files are compressed using gzip. If this field is unspecified, the value defaults to true.
max-size-mb

The maximum size in MiB that the file cache can use. If present, max-size-mb enables file caching in Cloud Storage FUSE and is useful if you want to limit the total capacity the Cloud Storage FUSE cache can use within its mounted directory.

  • Specify -1 to use the cache's entire available capacity in the directory you specify for cache-dir.
  • Specify 0 to disable the file cache.

If this field is unspecified, the value defaults to -1.
cache-file-for-range-read

A Boolean value that determines whether the full object should be downloaded asynchronously and stored in the Cloud Storage FUSE cache directory when the first read is done from a non-zero offset. This should be set to true if you plan on performing several random reads or partial reads. If this field is unspecified, the value defaults to false.

Note that if you do a partial read starting at offset 0, Cloud Storage FUSE always asynchronously downloads and caches the full object.
enable-parallel-downloads

Accelerates reads of large files by using the file cache directory as a prefetch buffer using multiple workers to download large files in parallel. The default value is false.


To use parallel downloads, you must enable file caching. To learn more about parallel downloads and configure its supporting properties, see Improve read performance using parallel downloads.
parallel-downloads-per-file

Specifies the number of maximum goroutines to spawn per file to download the object from Cloud Storage into the file cache. The default value is 16.
max-parallel-downloads

The maximum number of goroutines that can be spawned at any given time across all the download jobs of files. The default is twice the number of CPU cores on the machine. To bypass a limit, specify a value of -1

.
download-chunk-size-mb

Specifies the size of each read request in MiB that each goroutine makes to Cloud Storage when downloading the object into the file cache. The default is 50.
stat-cache-max-size-mb

The maximum size in MiB that the stat cache can use. The stat cache is always entirely kept in memory.

  • Specify 32 if your workload involves up to 20,000 files. If your workload is larger than 20,000 files, increase the size by values of 10 for every additional 6,000 files, which is an average of ~1,500 bytes per file.
  • Specify -1 to let the stat cache use as much memory as needed.
  • Specify 0 to disable the stat cache.

If this field is unspecified, the value defaults to 32.
ttl-secs

Defines the time to live (TTL) of cached metadata entries, expressed in seconds.

  • Specify -1 to bypass a TTL expiration and serve the file from the cache whenever it's available.
  • Specify 0 to ensure that the most up to date file is read. Using this value issues a Get metadata call to make sure that the object generation for the file in the cache matches what's stored in Cloud Storage. To learn more, see Configuring cache invalidation.

If this field is unspecified, the value defaults to 60.
type-cache-max-size-mb

The maximum size in MiB per directory that the type cache can use. The type cache is always entirely kept in memory.

  • Specify 4 if the maximum number of files within a single directory from the bucket you're mounting contains 20,000 files or less. If the maximum number of files within a single directory that you're mounting contains more than 20,000 files, increase the value by 1 for every 5,000 files, which is an average of ~200 bytes per file.
  • Specify -1 to let the type cache use as much memory as needed.
  • Specify 0 to disable the type cache.

If this field is unspecified, the value defaults to 4.
cache-dir Specifies the directory for storing file cache data. This field must be specified in order to enable file caching.
anonymous-access Disables authentication for requests. Set this field if you're using a custom endpoint that doesn't support authentication or if you're using Cloud Storage FUSE to mount public buckets. The default value is false.
kernel-list-cache-ttl-secs Enables the list cache and defines the time to live (TTL) in seconds of cached list entries. Memory allocation for the list cache is controlled by the kernel based on available memory. The default value is 0 which disables list caching.

To set the --kernel-list-cache-ttl-secs field, specify a positive value in seconds to keep the directory list response in the kernel's page cache. To bypass entry expiration and always return the list response from the cache when it's available, specify a value of -1.
ignore-interrupts Instructs Cloud Storage FUSE to ignore system interrupt signals, like SIGINT triggered by Control+C. This prevents signals from terminating in-flight operations. Values are true or false. The default value is true.