Creating an admin cluster (basic)

This page shows how to create an admin cluster.

The steps in this topic assume that you used gkeadm to create your admin workstation. If you did not use gkeadm and instead followed the advanced topic Creating an admin workstation with a static IP address, you might need to make adjustments as explained in each individual step. For a DHCP admin workstation, you must use the corresponding DHCP installation guide.

If you are behind a proxy, all gkectl commands automatically use the same proxy that you set in your configuration file for internet requests from the admin workstation. If your admin workstation is not located behind the same proxy, refer to the "Manual proxy options" in the advanced Creating an admin workstation topics: Static IP | DHCP.

SSH into your admin workstation

SSH into your admin workstation by following the instructions in Getting an SSH connection to your admin workstation.

Your allowlisted service account is activated on your admin workstation. Do all the remaining steps in this topic on your admin workstation.

Configuring static IPs for your admin cluster

To specify the static IP addresses that you want to use for your admin cluster, create a host configuration file named admin-hostconfig.yaml. For this exercise, you need to specify five IP addresses to be used by the admin cluster.

The following is an example of a host configuration file with five hosts:

hostconfig:
  dns: 172.16.255.1
  tod: 216.239.35.0
  otherdns:
  - 8.8.8.8
  - 8.8.4.4
  othertod:
  - ntp.ubuntu.com
  searchdomainsfordns:
  - "my.local.com"
blocks:
  - netmask: 255.255.252.0
    gateway: 172.16.23.254
    ips:
    - ip: 172.16.20.10
      hostname: admin-host1
    - ip: 172.16.20.11
      hostname: admin-host2
    - ip: 172.16.20.12
      hostname: admin-host3
    - ip: 172.16.20.13
      hostname: admin-host4
    - ip: 172.16.20.14
      hostname: admin-host5

The ips field is an array of IP addresses and hostnames. These are the IP addresses and hostnames that GKE on-prem will assign to your admin cluster nodes.

In the host configuration file, you also specify the addresses of the DNS servers, time servers, and default gateway that the admin cluster nodes will use.

The searchdomainsfordns field is an array of DNS search domains to use in the cluster. These domains are used as part of a domain search list.

Populated fields in your GKE on-prem configuration file

Recall that when you created your admin workstation, you filled in a configuration file named admin-ws-config.yaml. The gkeadm command-line tool used your admin-ws-config.yaml file to create the admin workstation.

When gkeadm created your admin workstation, it generated another configuration file namedadmin-cluster.yaml. This configuration file, which is on your admin workstation, is for creating your admin cluster.

The admin-ws-config.yaml and admin-cluster.yaml files have several fields in common. The values for those common fields are already populated in your admin-cluster.yaml file.

These are the fields that are already populated with default values and values that you provided when you created your admin workstation:

bundlePath:
vCenter:
  credentials:
    address:
    username:
    password:
  datastore:
  resourcePool:
  caCertPath:
  datacenter:
  cluster:
loadbalancer
  seesaw:
    cpus:
    memoryMB:
    enableHA:
gcrKeyPath:
stackdriver:
  projectID:
  serviceAccountKeyPath:

Leave all of the populated values unchanged.

Filling in the rest of your GKE on-prem configuration file

Next you need to fill in the remaining fields in your admin-cluster.yaml file.

vCenter.dataDisk

String. GKE on-prem creates a virtual machine disk (VMDK) to hold the Kubernetes object data for the admin cluster. The installer creates the VMDK for you, but you must provide a name for the VMDK in the vCenter.dataDisk field. For example:

vCenter:
  ...
  dataDisk: "my-disk.vmdk"

vSAN datastore: Creating a folder for the VMDK

If you are using a vSAN datastore, you need to put the VMDK in a folder. You must manually create the folder ahead of time. To do so, you could use govc to create a folder:

govc datastore.mkdir -namespace=true my-gke-on-prem-folder

Then set vCenter.dataDisk to the path of the VMDK, including the folder. For example:

vDenter:
...
dataDisk: "my-gke-on-prem-folder/my-disk.vmdk"

network.ipMode.type

String. Set this to "static".

network:
  ipMode:
    type: "static"

network.ipBlockFilePath

String. Because you are using static IP addresses, you must have a host configuration file as described in Configuring static IPs. Set network.ipBlockFilePath to the path of your host configuration file. For example:

network:
  ipBlockFilePath: "/my-config-directory/admin-hostconfig.yaml"

network.serviceCIDR and network.podCiDR

Strings. The admin cluster must have a range of IP addresses to use for Services and a range of IP addresses to use for Pods. These ranges are specified by the network.serviceCIDR and network.podCIDR fields. These fields are populated with default values`. If you like, you can change the populated values to values of your choice.

The Service and Pod ranges must not overlap. Also, the Service and Pod ranges must not overlap with IP addresses that are used for nodes in any cluster.

Example:

network:
  ...
  serviceCIDR: "10.96.232.0/24"
  podCIDR: "192.168.0.0/16"

network.vCenter.networkName

String. The name of the vSphere network for your cluster nodes.

If the name contains a special character, you must use an escape sequence for it.

If the network name is not unique, it is possible to specify a path to the network, such as /DATACENTER/network/NETWORK_NAME.

For example:

network:
  vCenter:
    networkName: "MY-CLUSTER-NETWORK"

loadBalancer.vips

Strings. Set the value of loadBalancer.vips.controlPlaneVIP to The IP address that you have chosen to configure on the load balancer for the Kubernetes API server of the admin cluster. Set the value of loadBalancer.vips.addonsVIP to the IP address you have chosen to configure on the load balancer for add-ons. For example:

loadBalancer:
  ...
  vips:
    controlplaneVIP: "203.0.113.3"
    addonsVIP: "203.0.113.4"

loadBalancer.kind

String. Set this to "Seesaw". For example:

loadBalancer:
  kind: "Seesaw"

loadBalancer.seesaw.ipBlockFilePath

String. Set this to the path of the hostconfig file for your Seesaw VM. For example:

loadbalancer:
  ...
  seesaw:
    ipBlockFilePath: "admin-seesaw-hostconfig.yaml"

loadBalancer.seesaw.vird

Integer. The virtual router identifier of your Seesaw VM. This identifier must be unique in a VLAN. Valid range is 1-255. For example:

loadBalancer:.
  seesaw:
    vrid: 125

loadBalancer.seesaw.cpus

Integer. The number of CPUs for your Seesaw VM. For example:

loadBalancer:.
  seesaw:
    cpus: 8

loadBalancer.seesaw.memoryMB

Integer. The number of megabytes of memory for your Seesaw VM. For example:

loadBalancer:.
  seesaw:
    memoryMB: 8192

loadBalancer.seesaw.masterIP

String. An IP address, of your choice, that your Seesaw VM will announce. For example:

loadBalancer:
  seesaw:
    masterIP: "172.16.20.21"

loadBalancer.seesaw.enableHA

Boolean. Set this to false. For example:

loadBalancer:.
  seesaw:
    enableHA: false

proxy

Strings. If your network is behind a proxy server, you must specify the proxy address and any addresses that should not go through the proxy server.

Set proxy.url to specify the HTTP address of your proxy server. You must include the port number even if it's the same as the scheme's default port. The proxy server you specify here is used by your GKE on-prem clusters. Also, your admin workstation is automatically configured to use this same proxy server unless you set the HTTPS_PROXY environment variable on your admin workstation.

Set proxy.noProxy to define a list of IP addresses, IP address ranges, host names, and domain names that should not go through the proxy server. When GKE on-prem sends a request to one of these addresses, hosts, or domains, that request is sent directly. For example:

proxy:
  url: "http://my-proxy.example.local:80"
  noProxy: "10.151.222.0/24, my-host.example.local,10.151.2.1"

stackdriver.clusterLocation

String. The Google Cloud region where you want to store logs. It is a good idea to choose a region that is near your on-prem data center. For example:

stackdriver:
  clusterLocation: "us-central1"

stackdriver.enableVPC

Boolean. Set stackdriver.enableVPC to true if you have your cluster's network controlled by a VPC. This ensures that all telemetry flows through Google's restricted IP addresses. Otherwise, set this to false. For example:

stackdriver:
  enableVPC: false

Additional fields in the configuration file

The GKE on-prem configuration file has several fields in addition to the ones shown in this topic. For example, you can use the manualLB field to configure GKE on-prem to run in manual load balancing mode.

For a complete description of the fields in the configuration file, see Installing using DHCP and Installing using static IP addresses.

Validating the configuration file

After you've modified the configuration file, run gkectl check-config to verify that the file is valid and can be used for installation:

gkectl check-config --config admin-cluster.yaml

If the command returns any FAILURE messages, fix the issues and validate the file again.

If you want to skip the more time-consuming validations, pass the --fast flag. To skip individual validations, use the --skip-validation-xxx flags. To learn more about the check-config command, see Running preflight checks.

Running gkectl prepare

Run gkectl prepare to initialize your vSphere environment:

gkectl prepare --config admin-cluster.yaml --skip-validation-all

Creating your load balancer

Create and configure the VM for your Seesaw load balancer:

gkectl create loadbalancer --config [CONFIG_FILE]

where [CONFIG_FILE] is the path of your configuration file.

Creating the admin cluster

Create your admin cluster::

gkectl create admin --config [CONFIG_FILE]

where [CONFIG_FILE] is the path of your configuration file.