Version 1.4. This version is no longer supported. For more information see the version support policy.

Proxy and firewall rules

Allowlisting addresses for your proxy

If your organization requires outbound traffic to pass through a proxy server, allowlist the following addresses in your proxy server. Note that www.googleapis.com is needed, instead of googleapis.com:

dl.google.com (required by Google Cloud SDK installer)
gcr.io
www.googleapis.com
accounts.google.com
cloudresourcemanager.googleapis.com
container.googleapis.com
gkeconnect.googleapis.com
gkehub.googleapis.com
iam.googleapis.com
logging.googleapis.com
monitoring.googleapis.com
oauth2.googleapis.com
servicecontrol.googleapis.com
serviceusage.googleapis.com
storage.googleapis.com
checkpoint-api.hashicorp.com
releases.hashicorp.com

If you use gkeadm to install GKE on-prem, you don't need to allowlist the hashicorp URLs above.

Also, if your vCenter Server has an external IP address, allowlist its address in your proxy server.

Firewall rules

Set up your firewall rules to allow the following traffic.

Firewall rules for IP addresses available in the admin cluster

The IP addresses available in the admin cluster are listed in the IP block file. These IP addresses are used for the admin cluster control-plane node, admin cluster add-on nodes, and the user cluster control-plane node. Because the IP addresses for the admin cluster are not assigned to specific nodes, you must make sure that all of the firewall rules listed in the following table apply to all of the IP addresses available for the admin cluster.

From	To	Port	Protocol	Description
Admin cluster control-plane node	vCenter Server API	443	TCP/https	Cluster resizing.
Admin cluster add-on nodes	vCenter Server API	443	TCP/https	User cluster lifecycle managemet.
User cluster control-plane node	vCenter Server API	443	TCP/https	Cluster resizing.
Cloud Logging Collector, which runs on an admin cluster add-on node	oauth2.googleapis.com logging.googleapis.com stackdriver.googleapis.com servicecontrol.googleapis.com	443	TCP/https
Cloud Monitoring Collector, which runs on an admin cluster add-on node	oauth2.googleapis.com monitoring.googleapis.com stackdriver.googleapis.com servicecontrol.googleapis.com	443	TCP/https
Admin cluster control-plane node	F5 BIG-IP API	443	TCP/https
User cluster control-plane node	F5 BIG-IP API	443	TCP/https
Admin cluster control-plane node	On-premises local Docker registry	Depends on your registry	TCP/https	Required if GKE on-prem is configured to use a local private Docker registry instead of gcr.io.
User cluster control-plane node	On-premises local Docker registry	Depends on your registry	TCP/https	Required if GKE on-prem is configured to use a local private Docker registry instead of gcr.io.
Admin cluster control-plane node	* gcr.io .googleusercontent.com .googleapis.com *.k8s.io	443	TCP/https	Download images from public Docker registries. Not required if using a private Docker registry. `*.googleusercontent.com` is the canonical DNS name for `gcr.io`.
User cluster control-plane node	* gcr.io .googleusercontent.com .googleapis.com *.k8s.io	443	TCP/https	Download images from public Docker registries. Not required if using a private Docker registry. `*.googleusercontent.com` is the canonical DNS name for `gcr.io`.
Admin cluster worker nodes	Admin cluster worker nodes	All	179 - bgp 443 - https 5473 - Calico/Typha 9443 - Envoy metrics 10250 - kubelet node port	All worker nodes must be layer-2 adjacent and without any firewall.
Admin cluster nodes	Admin cluster pod CIDR	all	any	External traffic get SNAT'ed on the first node and sent to pod IP.
Admin cluster worker nodes	User cluster nodes	22	ssh	API server to kubelet communication over an SSH tunnel.
Admin cluster nodes	IPs of Seesaw LB VMs of the admin cluster	20255,20257	TCP/http	LB config push and metrics monitoring. Only needed if you are using Bundled LB Seesaw.

Firewall rules for user cluster nodes

In the user cluster nodes, their IP addresses are listed in the IP block file.

As with the admin cluster nodes, you don't know which IP address will be used for which node. Thus, all of the rules in the user cluster nodes apply to each user cluster node.

From	To	Port	Protocol	Description
User cluster worker nodes	Admin workstation Docker registry
User cluster worker nodes	* gcr.io .googleusercontent.com .googleapis.com *.k8s.io	443	TCP/https	Download images from public Docker registries. Not required if using a private Docker registry. `*.googleusercontent.com` is the canonical DNS name for `gcr.io`.
User cluster worker nodes	F5 BIG-IP API	443	TCP/https
User cluster worker nodes	VIP of the pushprox server, which runs in the Admin cluster.	8443	TCP/https	Prometheus traffic.
User cluster worker nodes	User cluster worker nodes	all	22 - ssh 179 - bgp 443 - https 5473 - calico-typha 9443 - envoy metrics 10250 - kubelet node port"	All worker nodes must be layer-2 adjacent and without any firewall.
User cluster nodes	User cluster pod CIDR	all	any	External traffic get SNAT'ed on the first node and sent to pod IP.
Cloud Logging Collector, which runs on a random user cluster worker node	oauth2.googleapis.com logging.googleapis.com stackdriver.googleapis.com servicecontrol.googleapis.com	443	TCP/https
Connect agent, which runs on a random user cluster worker node.	gkeconnect.googleapis.com gkehub.googleapis.com www.googleapis.com oauth2.googleapis.com accounts.google.com	443	TCP/https	Connect traffic.
Cloud Monitoring Collector, which runs on a random user cluster worker node	oauth2.googleapis.com Monitoring.googleapis.com stackdriver.googleapis.com servicecontrol.googleapis.com	443	TCP/https
User cluster nodes	IPs of Seesaw LB VMs of the user cluster	20255,20257	TCP/http	LB config push and metrics monitoring. Only needed if you are using Bundled LB Seesaw.

Firewall rules for remaining components

These rules apply to all other components not listed in the tables for the admin cluster and user cluster nodes.

From	To	Port	Protocol	Description
Admin cluster pod CIDR	Admin cluster pod CIDR	all	any	Inter-pod traffic does L2 forwarding directly using source and destination IP within Pod CIDR.
Admin cluster pod CIDR	Admin cluster nodes	all	any	Return traffic of external traffic.
User cluster pod CIDR	User cluster pod CIDR	all	any	Inter-pod traffic does L2 forwarding directly using source and destination IP within Pod CIDR.
User cluster pod CIDR	User cluster nodes	all	any	Return traffic of external traffic.
Clients and application end users	VIP of Istio ingress	80, 443	TCP	End user traffic to the ingress service of a user cluster.
Jump server to deploy the admin workstation	checkpoint-api.hashicorp.com releases.hashicorp.com vCenter Server API ESXi VMkernel (mgt) IPs of hosts in target cluster	443	TCP/https	Terraform deployment of the admin workstation.
Admin workstation	* gcr.io .googleusercontent.com .googleapis.com *.k8s.io"	443	TCP/https	Download Docker images from public Docker registries. `*.googleusercontent.com` is the canonical DNS name for `gcr.io`.
Admin workstation	vCenter Server API F5 BIG-IP API	443	TCP/https	Cluster bootstrapping.
Admin workstation	ESXi VMkernel (mgt) IPs of hosts in target cluster	443	TCP/https	The admin workstation uploads the OVA to the datastore through the ESXi hosts.
Admin workstation	Node IP of Admin Cluster control-plane VM	443	TCP/https	Cluster bootstrapping.
Admin workstation	VIP of the admin cluster's Kubernetes API server VIPs of user clusters' Kubernetes API servers	443	TCP/https	Cluster bootstrapping. User cluster deletion.
Admin workstation	Admin cluster control-plane node and worker nodes	443	TCP/https	Cluster bootstrapping. Control plane upgrades.
Admin workstation	All admin cluster nodes and all user cluster nodes	443	TCP/https	Network validation as part of the `gkectl check-config` command.
Admin workstation	VIP of the admin cluster's Istio ingress VIP of user clusters' Istio ingress	443	TCP/https	Network validation as part of the `gkectl check-config` command.
Admin workstation	IPs of Seesaw LB VMs in both admin and user clusters Seesaw LB VIPs of both admin and user clusters	20256,20258	TCP/http/gRPC	Health check of LBs. Only needed if you are using Bundled LB Seesaw.
Admin workstation	Node IP of the admin cluster control plane	22	TCP	Required if you need SSH access from the admin workstation to the admin cluster control plane.
LB VM IPs	node IPs of the corresponding cluster	10256: node health check 30000 - 32767: healthCheckNodePort	TCP/http	Node health check. healthCheckNodePort is for services with externalTrafficPolicy set to Local. Only needed if you are using Bundled LB Seesaw.
F5 Self-IP	All admin and all user cluster nodes	30000 - 32767	any	For the data plane traffic that F5 BIG-IP load balances via a virtual server VIP to the node ports on the Kubernetes cluster nodes. Typically the F5 self-ip is on the same network/subnet as the Kubernetes cluster nodes.

< Previous Next >