This page provides details on how Cloud Storage FUSE automatically optimizes its default configuration settings when running on specific high-performance Google Cloud machine types.
Overview
Cloud Storage FUSE automatically optimizes default configuration settings when running on specific high-performance Google Cloud machine types to maximize performance for demanding, high-throughput workloads. Values that are manually set at the time of mount will override these defaults.
Machine types
Configurations are automated for the following high-performance machine types:
| Series type | Machine type |
|---|---|
| A2 machine series | |
a2-megagpu-16g |
|
a2-ultragpu-8g |
|
| A3 machine series | |
a3-edgegpu-8g |
|
a3-highgpu-8g |
|
a3-megagpu-8g |
|
a3-ultragpu-8g |
|
| A4 machine series | |
4-highgpu-8g-lowmem |
|
| TPU v5e | |
ct5l-hightpu-8t |
|
ct5lp-hightpu-8t |
|
| TPU v5p | |
ct5p-hightpu-4t |
|
ct5p-hightpu-4t-tpu |
|
| TPU v6e (Trillium) | |
ct6e-standard-4t |
|
ct6e-standard-4t-tpu |
|
ct6e-standard-8t |
|
ct6e-standard-8t-tpu |
Automated configurations
When a supported machine type is detected, Cloud Storage FUSE automatically applies the following configuration values:
| Configuration file field | CLI option | Automated configuration value |
|---|---|---|
metadata-cache.negative-ttl-secs |
--metadata-cache-negative-ttl-secs |
0 |
metadata-cache.ttl-secs1 |
--metadata-cache-ttl-secs1 |
|
metadata-cache.stat-cache-max-size-mb |
--stat-cache-max-size-mb |
1024 |
metadata-cache.type-cache-max-size-mb |
--type-cache-max-size-mb |
128 |
implicit-dirs |
--implicit-dirs |
true |
file-system.rename-dir-limit |
--rename-dir-limit |
200000 |
1Setting this configuration to -1 significantly boosts
performance by always serving files from the cache. Be aware that this
configuration bypasses consistency checks, which can lead to serving
outdated data. For details on managing data consistency, refer to
File, stat, and type cache invalidation.
Further performance tuning
When you use a high-performance Google Cloud machine type, the configurations detailed on this page are automatically applied. However, you can further fine-tune your machine for optimal performance using the Performance tuning best practices guide.
If you're running training, serving, or checkpointing and JIT cache workloads on Google Kubernetes Engine clusters that use Cloud GPUs or Cloud TPU to access large datasets in Cloud Storage, you can streamline your setup by utilizing pre-configured YAML files to mount your Cloud Storage buckets directly into your pods more efficiently. For more information and instructions on how to use pre-configured Google Kubernetes Engine YAML files, see Use pre-configured Google Kubernetes Engine YAML files to optimize Cloud Storage FUSE performance.
What's next
Learn how to tune Cloud Storage FUSE for optimal performance.
Use a pre-configured Google Kubernetes Engine YAML file to configure tuning best practices.