Overview of caching in Cloud Storage FUSE

This page provides details on caching options available for Cloud Storage FUSE and how each cache type can be configured.

To help increase the performance of data retrieval, Cloud Storage FUSE offers four types of optional caching:

Considerations

  • Enabling file caching, stat caching, type caching, or list caching can increase performance but reduce consistency, which usually occurs when you access the same bucket using multiple clients with a high change rate. To reduce the impact on consistency, we recommend mounting buckets as read-only. To learn more about caching behavior, see Cloud Storage FUSE semantics documentation on GitHub.

  • If a file cache entry hasn't yet expired based on its time to live (TTL) and the file is in the cache, read operations to that file are served from the local client cache without any request being issued to Cloud Storage.

  • If a file cache entry has expired based on its TTL, a GET metadata call is first made to Cloud Storage, and if the file isn't in the cache, the file is retrieved from Cloud Storage. Both operations are subject to network latencies. If the metadata entry has been invalidated, but the file is in the cache, and its object generation has not changed, the file is served from the cache only after the GET metadata call is made to check if the data is valid.

  • If a Cloud Storage FUSE client modifies a cached file or its metadata, then the file is immediately invalidated and consistency is ensured in the following read by the same client. However, if different clients access the same file or its metadata, and its entries are cached, then the cached version of the file or metadata is read and not the updated version until the file is invalidated by that specific client's TTL setting.

  • To avoid cache thrashing, ensure that your entire dataset fits into the cache capacity. Also, consider the maximum capacity and performance that your cache media can provide. If you hit the provisioned cache's maximum performance, capacity limit, or both, it's beneficial to read directly from Cloud Storage which has much higher limits than Cloud Storage FUSE.

Read path for cached data

The Cloud Storage FUSE cache accelerates repeat reads after they've been ingested to the cache. Both first-time reads and cache misses go directly to Cloud Storage and are subject to normal Cloud Storage network latencies. To improve first-time read performance, see Pre-populate the metadata cache.

File caching overview

The Cloud Storage FUSE file cache is a client-based read cache that serves repeat file reads from cache storage backed by your local file system. To learn more about the file caching, see the File caching page.

Parallel downloads

When you enable file caching, parallel downloads are enabled automatically on Cloud Storage FUSE versions 2.12 and later. Parallel downloads accelerate read performance for large files over 1 GB in size including first-time reads, using multiple workers to download a file in parallel using the file cache directory as a prefetch buffer. Parallel downloads can be used to optimize performance for artificial intelligence (AI) and machine learning (ML) workloads, such as model serving, checkpoint restores, and training on large objects. For more information about parallel downloads, see Parallel downloads.

Configure file caching

To learn how to configure file caching, see Use Cloud Storage FUSE file caching.

Stat caching overview

The Cloud Storage FUSE stat cache is a cache for object metadata that improves performance for operations specific to file attributes such as size, modification time, or permissions. Using stat cache improves latency by using cached data to perform operations instead of sending a stat object request to Cloud Storage. To learn more about stat caching, see the Semantics documentation on GitHub.

Configure stat cache

The stat cache is enabled by default and can be configured using a Cloud Storage FUSE configuration file. The maximum size of the cache is controlled using one of the following methods:

For more information about default and recommended values for configuring, the stat cache, see the Cloud Storage FUSE CLI options and configuration file pages.

Configure negative stat caching

Cloud Storage FUSE also offers a negative stat caching capability, which lets you set a TTL for stat cache entries for non-existent files, or negative results. The TTL of the negative stat cache is controlled using one the following methods:

Type caching overview

The Cloud Storage FUSE type cache is a metadata cache that accelerates performance for metadata operations specific to file or directory existence. Using type cache improves latency by reducing the number of requests made to Cloud Storage to check if a file or directory exists by storing this information locally. To learn more about type caching, see the Semantics documentation on GitHub.

Configure type caching

The type cache is enabled by default and can be configured using a Cloud Storage FUSE configuration file or the Cloud Storage FUSE CLI. The maximum size of the cache is controlled using one of the methods:

For more information about type caching and recommended values, see the CLI options or configuration file pages.

The TTL of the cache is controlled using one of the following methods:

List caching overview

The Cloud Storage FUSE list cache is for directory and file list, or ls, responses that improves list operation speeds. List caching is especially useful for workloads that repeat full directory listings as part of execution, such as AI/ML training runs.

The list cache is kept in memory in the page cache, which is controlled by the kernel based on memory availability, as opposed to the stat and type caches, which are kept in your machine's memory and controlled by Cloud Storage FUSE.

Configure list caching

You can enable list caching using one of the following methods:

To enable and configure list caching, see the Cloud Storage FUSE configuration file or the Cloud Storage FUSE CLI options pages.

Configure file, stat, and type cache invalidation

The following sections describe how to configure cache invalidation for all cache types.

For file, stat, and type caches, use one of the following methods to specify the TTL in seconds for how long cached metadata is used from when it's fetched from Cloud Storage to when it expires and needs to be refreshed:

When you specify a value that's greater than 0, the metadata for the file cache remains valid only for the amount of time you specified. For file caching, we recommend increasing the value based on the expected time between repeat reads while you balance consistency needs. Based on the importance and frequency of the data changing, we recommend setting the value as high as your workload lets you. When a metadata entry becomes invalid, subsequent reads are queried from Cloud Storage.

In addition to accepting values that represent a specific TTL in seconds before your cached metadata expires and needs to be refreshed, you can use the following values to specify how your file is read:

  • 0: ensures the file with the most up to date data is read by issuing a GET metadata call to Cloud Storage that checks the file it's serving from to ensure the cache is consistent. If the file in the cache is up to date, it's served directly from the cache. Specifying a value other than 0 can lead to reduced performance because a call must always be made to Cloud Storage to check the metadata first. If the file is in the cache and hasn't changed, the file is served from the cache with consistency after the GET metadata call.

  • -1: ensures the file is always read from the cache if it's available, without checking for consistency. Serving files without checking for consistency can serve inconsistent data, and should only be used temporarily for workloads that run in jobs with non-changing data. For example, using a value of -1 is useful for machine learning training, where the same data is read across multiple epochs without changes.

List cache invalidation

List cache invalidation is set by specifying a value greater than 0 using one of the following methods:

  • --kernel-list-cache-ttl-secs CLI option
  • file-system:kernel-list-cache-ttl-secs field

The directory list response is kept in the kernel's page cache and remains valid for the amount of time you specified. When you specify a value of -1, Cloud Storage FUSE disables list cache expiration and returns the list response from the cache when it's available. Specifying a value of 0 disables the list cache.

What's next