This topic outlines the Kubernetes Cluster API resources used in GKE On-Prem.
Overview
GKE On-Prem supports the following Cluster API resources:
- Cluster
- Machine
- MachineDeployment
- MachineSet
- MachineClasses
GKE On-Prem automatically generates these resources in your clusters during installation. Generally, you don't need to interact directly with them, apart from advanced procedures like manually deleting a user cluster. Additionally, GKE On-Prem documentation references these resources to illustrate concepts like high-availability.
You can check which Cluster API CustomResourceDefinitions (CRDs) are available in a cluster:
kubectl get crds | grep "cluster.k8s.io" clusters.cluster.k8s.io machineclasses.cluster.k8s.io machinedeployments.cluster.k8s.io machines.cluster.k8s.io machinesets.cluster.k8s.io
The following sections discuss the Cluster API resources currently relevant to understanding GKE On-Prem.
Cluster
A Cluster resource represents an instance of Kubernetes running on a host. All GKE On-Prem clusters have a Cluster resource that describes the cluster.
To see Cluster resources in a cluster, run the following command, where [KUBECONFIG] is the path to the cluster's kubeconfig file:
kubectl get clusters --kubeconfig [KUBECONFIG]
Below is an example of a Cluster resource that you might see in a GKE On-Prem user cluster:
apiVersion: cluster.k8s.io/v1alpha1
kind: Cluster
metadata:
annotations:
addons.gke.io/loadbalancerip: ""
bundle.gke.io/target-version:
bundle.gke.io/version:
onprem.gke.io/provider: vsphere
creationTimestamp:
finalizers:
- cluster.cluster.k8s.io
generation:
name: example-user-cluster
namespace: default
resourceVersion: ""
selfLink: /apis/cluster.k8s.io/v1alpha1/namespaces/default/clusters/example-user-cluster
uid:
spec:
clusterNetwork:
pods:
cidrBlocks:
-
serviceDomain: cluster.local
services:
cidrBlocks:
-
providerSpec:
value:
apiVersion: vsphereproviderconfig.k8s.io/v1alpha1
controlPlaneReplicas:
controlPlaneVersion:
kind: VsphereClusterProviderConfig
loadBalancerIP:
loadBalancerNodePort:
metadata:
creationTimestamp:
networkSpec:
dns:
ntp: ""
reservedAddresses:
oidcSpec:
sniCert:
vsphereDatacenter:
vsphereDefaultDatastore:
vsphereNetwork:
vsphereResourcePool:
status:
apiEndpoints:
- host:
port:Machine
In GKE On-Prem, a Machine resource is a declarative specification for an individual virtual machine (VM) that lives in vSphere. Each Machine resource has a corresponding Node resource that is created by GKE On-Prem based on the Machine's specification. These resources reference the same physical object: a VM living in your vSphere environment.
To see Machine resources in a cluster, run the following command, where [KUBECONFIG] is the path to the cluster's kubeconfig file:
kubectl get machines --kubeconfig [KUBECONFIG]
Below is an example of a Machine resource that you might see in a GKE On-Prem cluster:
apiVersion: cluster.k8s.io/v1alpha1
kind: Machine
metadata:
annotations:
control-plane-version: ""
kubelet-version:
onprem.gke.io/provider: vsphere
vm-ip-address:
vsphere-machine-state:
vsphere-machine-uuid:
creationTimestamp:
finalizers:
- machine.cluster.k8s.io
generateName: example-cluster-node
generation:
labels:
kubernetes.googleapis.com/cluster-name: example-cluster-node
kubernetes.googleapis.com/cluster-namespace:
machine-template-hash: ""
set: node
name: example-cluster-node
namespace: default
ownerReferences:
- apiVersion: cluster.k8s.io/v1alpha1
blockOwnerDeletion:
controller:
kind: MachineSet
name: example-cluster-node
uid:
resourceVersion: ""
selfLink: /apis/cluster.k8s.io/v1alpha1/namespaces/default/machines/example-cluster-node
uid:
spec:
metadata:
creationTimestamp:
providerSpec:
value:
apiVersion: vsphereproviderconfig.k8s.io/v1alpha1
kind: VsphereMachineProviderConfig
machineVariables:
datacenter:
datastore:
disk_label:
disk_size: ""
memory: ""
network:
num_cpus: ""
resource_pool:
vm_template:
metadata:
creationTimestamp:
networkSpec:
address:
dns:
ntp: ""
useIPAM:
vsphereMachine:
versions:
kubelet:
status:
lastUpdated:
nodeRef:
kind: Node
name:
uid:MachineDeployment
A MachineDeployment manages a group of Machines: it specifies the GKE On-Prem cluster in which the Machines run, declares their vSphere configuration, and rolls out changes to configurations. You can think of MachineDeployments and Machines being like Deployments and Pods.
To see MachineDeployment resources in a cluster, run the following command, where [KUBECONFIG] is the path to the cluster's kubeconfig file:
kubectl get machinedeployments --kubeconfig [KUBECONFIG]
Below is an example of a MachineDeployment resource that you might see in a GKE On-Prem cluster:
apiVersion: cluster.k8s.io/v1alpha1
kind: MachineDeployment
metadata:
annotations:
machinedeployment.clusters.k8s.io/revision: "1"
onprem.gke.io/provider: vsphere
creationTimestamp:
generation: 2
labels:
kubernetes.googleapis.com/cluster-name: example-cluster
kubernetes.googleapis.com/cluster-namespace:
set: node
name: example-cluster
namespace: default
resourceVersion: ""
selfLink: /apis/cluster.k8s.io/v1alpha1/namespaces/default/machinedeployments/example-cluster
uid:
spec:
minReadySeconds: 0
progressDeadlineSeconds:
replicas:
revisionHistoryLimit:
selector:
matchLabels:
kubernetes.googleapis.com/cluster-name: example-cluster
kubernetes.googleapis.com/cluster-namespace: default
set: node
strategy:
rollingUpdate:
maxSurge:
maxUnavailable:
type: RollingUpdate
template:
metadata:
annotations:
onprem.gke.io/provider: vsphere
creationTimestamp: null
labels:
kubernetes.googleapis.com/cluster-name: example-cluster
kubernetes.googleapis.com/cluster-namespace:
set: node
spec:
metadata:
creationTimestamp:
providerSpec:
value:
apiVersion: vsphereproviderconfig.k8s.io/v1alpha1
kind: VsphereMachineProviderConfig
machineVariables:
datacenter:
datastore:
disk_label:
disk_size: ""
memory: ""
network:
num_cpus: ""
resource_pool: example-cluster
vm_template:
metadata:
creationTimestamp:
networkSpec:
address:
dns:
ntp: ""
useIPAM:
vsphereMachine:
versions:
kubelet:
status:
availableReplicas:
observedGeneration:
readyReplicas:
replicas:
updatedReplicas: