Parallelstore is available by invitation only. If you'd like to request
access to Parallelstore in your Google Cloud project, contact your
sales representative.
Parallelstore is a fully managed, low-latency
distributed file system designed to meet the demands of AI/ML training and
high performance computing (HPC) workloads that need extremely low latency
(sub-millisecond), full POSIX semantics, and high
metadata operation throughput. Parallelstore scales to 1 TB/s read speeds
and millions of IOPS.
To connect a Google Kubernetes Engine (GKE) cluster to a Parallelstore instance,
use the Parallelstore Container Storage Interface (CSI) driver. The
Parallelstore CSI driver lets you use the GKE API to consume
Parallelstore instances as volumes for your stateful workloads (for example,
Pods and Jobs). It's optimized for AI/ML training workloads, particularly those
involving smaller file sizes and random reads.
GKE enables the CSI driver for you by default when you create a
new GKE Autopilot cluster. On new and existing
GKE Standard clusters, you'll need to enable the
CSI driver.
Benefits
You can use the Parallelstore CSI driver to benefit from high-performance storage.
With the Parallelstore CSI driver, you can accelerate your high performance
computing and AI/ML training workloads, with fast, consistent access to shared
data for efficient processing and analysis.
You have access to fully-managed parallel file systems as your storage
through the
Kubernetes APIs.
The Google Kubernetes Engine Parallelstore CSI driver supports the ReadWriteMany,
ReadOnlyMany, and ReadWriteOnceaccess modes.
You can use the Google Kubernetes Engine Parallelstore CSI driver to dynamically provision
your PersistentVolumes.
You can access existing Parallelstore instances in Kubernetes workloads. You
can also dynamically create Parallelstore instances and use them in Kubernetes
workloads with a
StatefulSet
or a
Deployment.
Limitations
Data persistence: Parallelstore is a "scratch plus" file system. It's
backed by Local SSD with 2+1 erasure coding, and the mean time to data loss is
two months. Parallelstore is not long-term storage and should instead be
considered an extremely fast file system for specific workloads.
Per-Pod limitation: GKE supports mounting only one
Parallelstore instance per Pod.
Note: Starting with GKE version 1.32.3,
you can use the node mount
feature of the Parallelstore CSI driver to mount multiple Parallelstore
instances per Pod.
Data transfers: Transferring data from Cloud Storage to Parallelstore is
not supported by the GKE API. To perform the transfer, use the
Parallelstore API.
Note: Starting with GKE version 1.31.1,
you can use the GKE Volume Populator
to transfer data from Cloud Storage to Parallelstore during dynamic provisioning.
Usable capacity: You can configure storage capacity from 12,000 GiB
to 100,000 GiB.
Supported zones: Parallelstore is supported in
these zones. If the region of
your cluster differs from that of your Parallelstore instance, there will be a
noticeable decline in the I/O performance.
VPC-SC limitations for Parallelstore: If you use both Shared VPC and
VPC Service Controls, you must have the host project that provides the network and
the service project that contains the Parallelstore instance inside the same
perimeter for the Parallelstore instance to function correctly. Separating the
host project and service project with a perimeter might cause the existing
instances to become unavailable and might not create new instances.
ARM machine support: The Parallelstore CSI Driver supports only
amd64 architecture nodes. ARM-based nodes are not supported at this time.
Requirements
To use the Parallelstore CSI driver, your clusters must meet the following
requirements:
Make sure you have installed the latest version of the Google Cloud CLI. The minimum supported gcloud CLI
version for this feature is 469.0.0 or later.
Use Google Kubernetes Engine cluster version 1.29 or later.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-26 UTC."],[],[],null,["# About the Google Kubernetes Engine Parallelstore CSI driver\n\n[Autopilot](/kubernetes-engine/docs/concepts/autopilot-overview) [Standard](/kubernetes-engine/docs/concepts/choose-cluster-mode)\n\n*** ** * ** ***\n\nParallelstore is available by invitation only. If you'd like to request access to Parallelstore in your Google Cloud project, contact your sales representative.\n\n[Parallelstore](/parallelstore/docs/overview) is a fully managed, low-latency\ndistributed file system designed to meet the demands of AI/ML training and\nhigh performance computing (HPC) workloads that need extremely low latency\n(sub-millisecond), full POSIX semantics, and high\nmetadata operation throughput. Parallelstore scales to 1 TB/s read speeds\nand millions of IOPS.\n\nTo connect a Google Kubernetes Engine (GKE) cluster to a Parallelstore instance,\nuse the Parallelstore Container Storage Interface (CSI) driver. The\nParallelstore CSI driver lets you use the GKE API to consume\nParallelstore instances as volumes for your stateful workloads (for example,\nPods and Jobs). It's optimized for AI/ML training workloads, particularly those\ninvolving smaller file sizes and random reads.\n\nGKE enables the CSI driver for you by default when you create a\nnew GKE Autopilot cluster. On new and existing\nGKE Standard clusters, you'll need to enable the\nCSI driver.\n\nBenefits\n--------\n\nYou can use the Parallelstore CSI driver to benefit from high-performance storage.\nWith the Parallelstore CSI driver, you can accelerate your high performance\ncomputing and AI/ML training workloads, with fast, consistent access to shared\ndata for efficient processing and analysis.\n\n- You have access to fully-managed parallel file systems as your storage through the [Kubernetes APIs](https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-api).\n- The Google Kubernetes Engine Parallelstore CSI driver supports the **ReadWriteMany** , **ReadOnlyMany** , and **ReadWriteOnce** [access modes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes).\n- You can use the Google Kubernetes Engine Parallelstore CSI driver to dynamically provision your PersistentVolumes.\n- You can access existing Parallelstore instances in Kubernetes workloads. You can also dynamically create Parallelstore instances and use them in Kubernetes workloads with a [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) or a [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/).\n\nLimitations\n-----------\n\n- **Data persistence**: Parallelstore is a \"scratch plus\" file system. It's\n backed by Local SSD with 2+1 erasure coding, and the mean time to data loss is\n two months. Parallelstore is not long-term storage and should instead be\n considered an extremely fast file system for specific workloads.\n\n- **Per-Pod limitation**: GKE supports mounting only one\n Parallelstore instance per Pod.\n\n**Note:** Starting with GKE version 1.32.3, you can use the [node mount](/parallelstore/docs/connect-from-kubernetes-engine#node-mount) feature of the Parallelstore CSI driver to mount multiple Parallelstore instances per Pod.\n\n- **Data transfers**: Transferring data from Cloud Storage to Parallelstore is not supported by the GKE API. To perform the transfer, use the Parallelstore API.\n\n**Note:** Starting with GKE version 1.31.1, you can use the [GKE Volume Populator](/kubernetes-engine/docs/how-to/persistent-volumes/volume-populator) to transfer data from Cloud Storage to Parallelstore during dynamic provisioning.\n\n- **Usable capacity**: You can configure storage capacity from 12,000 GiB to 100,000 GiB.\n- **Supported zones** : Parallelstore is supported in [these zones](/parallelstore/docs/locations). If the region of your cluster differs from that of your Parallelstore instance, there will be a noticeable decline in the I/O performance.\n- **VPC-SC limitations for Parallelstore**: If you use both Shared VPC and VPC Service Controls, you must have the host project that provides the network and the service project that contains the Parallelstore instance inside the same perimeter for the Parallelstore instance to function correctly. Separating the host project and service project with a perimeter might cause the existing instances to become unavailable and might not create new instances.\n- **ARM machine support**: The Parallelstore CSI Driver supports only amd64 architecture nodes. ARM-based nodes are not supported at this time.\n\nRequirements\n------------\n\nTo use the Parallelstore CSI driver, your clusters must meet the following\nrequirements:\n\n- Make sure you have installed the latest version of the [Google Cloud CLI](/sdk/gcloud#download_and_install_the). The minimum supported gcloud CLI version for this feature is 469.0.0 or later.\n- Use Google Kubernetes Engine cluster version 1.29 or later.\n\nWhat's next\n-----------\n\n- [Create and use a volume backed by a Parallelstore instance Google Kubernetes Engine](/kubernetes-engine/docs/how-to/persistent-volumes/parallelstore-csi-new-volume).\n- [Access an existing Parallelstore instance from Google Kubernetes Engine](/kubernetes-engine/docs/how-to/persistent-volumes/parallelstore-csi-existing-instance)."]]