Overview: A highly scalable, highly durable, low cost
object store. It's suitable for storing vast datasets required for
training and model checkpoints, as well as hosting the final trained
models. Cloud Storage with Cloud Storage FUSE is the recommended
storage solution for most AI and ML use cases because it lets you scale
your data storage with more cost efficiency than file system services.
Supports large scale (up to EBs) training data for GPU and TPU
clusters.
Supports high-throughput (up to 1.25TB/s bandwidth or greater). To
maximize your throughput in Cloud Storage,
request more bandwidth.
Through integration with Cloud Storage FUSE,
Cloud Storage buckets can be mounted as local file systems. The
Cloud Storage FUSE CSI driver
also lets you mount buckets as local file systems in
Google Kubernetes Engine (GKE) for scaled AI and ML workloads.
Use Anywhere Cache
to co-locate storage in the same zone as compute workloads, providing
higher throughput (up to 2.5TB/s), lower latency, and location
flexibility when used with a multi-region bucket.
Overview: A high performance, fully managed parallel file system
optimized for AI and high performance computing (HPC) applications.
Suited for environments in which multiple compute nodes need fast and
consistent access to shared data for simulations, modeling, and
analysis.
Scales to 1 PB (936 TiB) capacity and up to 1 TB/s of throughput.
Supports thousands of IOPS/TiB.
Delivers ultra low sub-ms latency.
Has full POSIX support which enables out of the box migration of
on-premises AI workloads to Google Cloud.
Overview: An easy-to-use managed network-attached storage (NAS)
that's available over a Network File System (NFS) mount. Optimal for a
wide range of enterprise workloads where extreme parallel performance
isn't the primary driver. Filestore is not recommended for
large-scale HPC or AI and ML workloads.
Scales to dozens of clients and 100TB of capacity.
Has full POSIX support which enables out of the box migration of
on-premises AI workloads to Google Cloud.
Recommended for:
Home directories with small-scale development and test
environments
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-05-07 UTC."],[[["Storage services are crucial for various AI and ML workloads, including loading model binaries, storing checkpoints, and managing training data."],["The optimal storage solution depends on the specific use case, with options ranging from high durability for model binaries to low durability for temporary data."],["Filestore (Zonal tier) is ideal for small-scale AI/ML training and serving, providing an NFS mount and scaling to dozens of clients."],["Cloud Storage, with or without Cloud Storage FUSE, supports large-scale training data, offering high scalability, durability, and throughput."],["Parallelstore and Sycomp Storage Scale (GPFS) deliver high-performance, low-latency solutions for scratch data, model serving, and large training datasets, with features like POSIX support and on-premises data caching."]]],[]]
