Google Dataproc uses Ubuntu, Debian, and Rocky Linux image versions to bundle operating system, big data components, and Google Cloud connectors into one package that is deployed on a cluster. For more information, see Dataproc Versioning.
Notes:
- Dataproc image versions are supported for 24 months after their initial release.
- Generally, Dataproc image versions are available for 24 months after their end-of-support date, but an availability period can shorten if an image support date is extended.
Default Dataproc image version
Dataproc updates the default image version to the latest generally available Debian-based Dataproc after its General availability (GA) release date.
Supported Dataproc image versions
Debian images
The following Debian-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.
| Version | Last Updated | Released On | Supported Until | Available Until | Notes |
|---|---|---|---|---|---|
| 2.2-debian12 | 2024/10/31 | 2023/12/08 | 2025/12/31 | 2027/12/31 | General availability release. Image version 2.2 becomes the default image version on September, 13, 2024. |
| 2.1-debian11 | 2024/10/31 | 2022/12/12 | 2025/06/31 | 2026/12/31 | General availability release. |
| 2.0-debian10 | 2024/10/31 | 2021/01/22 | 2025/06/31 | 2026/07/31 | General availability release. |
Ubuntu images
The following Ubuntu LTS-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.
| Version | Last Updated | Released On | Supported Until | Available Until | Notes |
|---|---|---|---|---|---|
| 2.2-ubuntu22 | 2024/10/31 | 2023/12/08 | 2025/12/31 | 2027/12/31 | General availability release. |
| 2.1-ubuntu20 | 2024/10/31 | 2022/12/12 | 2025/06/31 | 2026/12/31 | General availability release. |
| 2.0-ubuntu18 | 2024/10/31 | 2021/01/22 | 2025/06/31 | 2026/07/31 | General availability release. |
Rocky Linux images
The following Rocky Linux-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.
| Version | Last Updated | Released On | Supported Until | Available Until | Notes |
|---|---|---|---|---|---|
| 2.2-rocky9 | 2024/10/31 | 2023/12/08 | 2025/12/31 | 2027/12/31 | General availability release. |
| 2.1-rocky8 | 2024/10/31 | 2022/12/12 | 2025/06/31 | 2026/12/31 | General availability release. |
| 2.0-rocky8 | 2024/10/31 | 2022/02/18 | 2025/06/31 | 2026/07/31 | General availability release. |
Unsupported Dataproc image versions
The following Dataproc versions are unsupported. Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.
| Version | Includes | Released On | Last Updated | Notes |
|---|---|---|---|---|
1.5-debian10/-ubuntu18/-rocky8 |
Apache Spark 2.4.8 Apache Hadoop 2.10.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 2.1.9-hadoop2 Python 3.7 Scala 2.12.10 Zookeeper 3.4.14 |
2020/03/25:debian10/ubuntu18 2022/02/18:rocky8 |
2023/04/28 | Unsupported as of 2023/04/28. 1.5.89-debian10/-ubuntu18/-rocky8 was the final released version. |
2.0-centos8 |
Apache Spark 3.1.2 Apache Hadoop 3.2.2 Apache Pig 0.18.0-SNAPSHOT Apache Hive 3.1.2 Cloud Storage connector 2.2.4-hadoop3 Python 3.8 Scala 2.12.14 Zookeeper 3.4.14 |
2021/03/16 | 2022/02/01 | Unsupported as of 2022/02/01. 2.0.30-centos8 was the final released version. |
1.5-centos8 |
Apache Spark 2.4.8 Apache Hadoop 2.10.1 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 2.1.5-hadoop2 Python 3.7 Scala 2.12.10 Zookeeper 3.4.14 |
2020/12/14 | 2022/02/01 | Unsupported as of 2022/02/01. 1.5.56-centos8 was the final released version. |
1.4-debian10/-ubuntu18 |
Apache Spark 2.4.8 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.18-hadoop2 Python 3.6 Scala 2.11.12 Zookeeper 3.4.14 |
2019/03/22 | 2022/02/01 | Unsupported as of 2022/02/01. 1.4.80-debian10/-ubuntu18 was the final released version. |
1.3-debian10/-ubuntu18 |
Apache Spark 2.3.4 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.18-hadoop2 Python 2.7 Scala 2.11.8 Zookeeper 3.4.13 |
2018/06/29 | 2021/12/22 | Unsupported as of 2021/08/01. 1.3.95-debian10/-ubuntu18 was the final released version, which has log4j2 vulnerabilities addressed. Note: previously released versions are vulnerable and must be upgraded. |
1.4-debian9 |
Apache Spark 2.4.5 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.17-hadoop2 Python 3.6 Scala 2.11.12 Zookeeper 3.4.13 |
2019/03/22 | 2020/07/10 | Unsupported as of 2020/07/10. 1.4.33-debian9 was the final released version. |
1.3-debian9 |
Apache Spark 2.3.4 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.17-hadoop2 Python 2.7 Scala 2.11.8 Zookeeper 3.4.13 |
2018/06/29 | 2020/07/10 | Unsupported as of 2020/07/10. 1.3.62-debian9 was the final released version. |
1.2-debian9 |
Apache Spark 2.2.3 Apache Hadoop 2.8.5 Apache Pig 0.16.0 Apache Hive 2.1.1 Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2 Python 2.7 Scala 2.11.8 Zookeeper 3.4.13 |
2017/07/21 | 2020/07/10 | Unsupported as of 2020/07/10. 1.2.102-debian9 was the final released version. |
1.1-debian9 |
Apache Spark 2.0.2 Apache Hadoop 2.7.7 Apache Pig 0.16.0 Apache Hive 2.1.1 Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2 |
2016/08/08 | 2019/09/26 | Unsupported as of 2019/10/01. 1.1.121-debian9 is the final released version. |
1.0-debian9 |
Apache Spark 1.6.2 Apache Hadoop 2.7.4 Apache Pig 0.15.0 Apache Hive 1.2.1 Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2 |
2016/02/22 | 2019/05/09 | GA image first release. Unsupported as of 2019/04/01. 1.0.119-debian9 was the final released version. |
0.2 |
Apache Spark 1.5.2 Apache Hadoop 2.7.1 Apache Pig 0.15.0 Apache Hive 1.2.1 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2 |
2015/11/18 | 2016/08/02 | Beta image second release. |
0.1 |
Apache Spark 1.5.0 Apache Hadoop 2.7.1 Apache Pig 0.14.10 Apache Hive 1.0 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2 |
2015/09/23 | 2016/08/02 | Dataproc beta release. Spark 1.5 has been compiled against Hive 1.2. |